Note: This is test shard 8 of 8.
[==========] Running 7 tests from 6 test suites.
[----------] Global test environment set-up.
[----------] 1 test from TableRangeRebalancingTest
[ RUN ] TableRangeRebalancingTest.Basic
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20240502 07:58:07.589763 3629 test_util.cc:268] Using random seed: -156319373
I20240502 07:58:07.595835 3629 ts_itest-base.cc:112] Starting cluster with:
I20240502 07:58:07.595927 3629 ts_itest-base.cc:113] --------------
I20240502 07:58:07.595979 3629 ts_itest-base.cc:114] 3 tablet servers
I20240502 07:58:07.596016 3629 ts_itest-base.cc:115] 3 replicas per TS
I20240502 07:58:07.596027 3629 ts_itest-base.cc:116] --------------
2024-05-02T07:58:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-02T07:58:07Z Disabled control of system clock
I20240502 07:58:07.604518 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.139.126:37175
--webserver_interface=127.3.139.126
--webserver_port=0
--builtin_ntp_servers=127.3.139.84:32841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.139.126:37175
--tserver_unresponsive_timeout_ms=3000
--enable_range_replica_placement=false
--raft_prepare_replacement_before_eviction=true
--enable_range_replica_placement=false with env {}
W20240502 07:58:07.700629 3641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:07.700825 3641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:07.700862 3641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:07.703708 3641 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:58:07.703758 3641 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240502 07:58:07.703781 3641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:07.703805 3641 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240502 07:58:07.703821 3641 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240502 07:58:07.707187 3641 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:32841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.3.139.126:37175
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.139.126:37175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.3.139.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.3641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:07.707906 3641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:07.708847 3641 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:07.715899 3647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:07.716210 3646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:07.716554 3649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:07.716948 3641 server_base.cc:1034] running on GCE node
I20240502 07:58:07.717355 3641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:07.718156 3641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:07.719295 3641 hybrid_clock.cc:648] HybridClock initialized: now 1714636687719287 us; error 25 us; skew 500 ppm
I20240502 07:58:07.721021 3641 webserver.cc:460] Webserver started at http://127.3.139.126:46595/ using document root <none> and password file <none>
I20240502 07:58:07.721524 3641 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:07.721580 3641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:07.721736 3641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:07.723248 3641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "5cc07f4a061c4d4dad410153843ce3a5"
format_stamp: "Formatted at 2024-05-02 07:58:07 on dist-test-slave-twcr"
I20240502 07:58:07.723671 3641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "5cc07f4a061c4d4dad410153843ce3a5"
format_stamp: "Formatted at 2024-05-02 07:58:07 on dist-test-slave-twcr"
I20240502 07:58:07.726802 3641 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:07.729342 3655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:07.730554 3641 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20240502 07:58:07.730669 3641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal
uuid: "5cc07f4a061c4d4dad410153843ce3a5"
format_stamp: "Formatted at 2024-05-02 07:58:07 on dist-test-slave-twcr"
I20240502 07:58:07.730751 3641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:07.747442 3641 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:07.747922 3641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:07.748032 3641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:07.753793 3641 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.126:37175
I20240502 07:58:07.753800 3707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.126:37175 every 8 connection(s)
I20240502 07:58:07.754597 3641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240502 07:58:07.756826 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:07.758956 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 3641
I20240502 07:58:07.759142 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240502 07:58:07.762070 3708 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cc07f4a061c4d4dad410153843ce3a5: Bootstrap starting.
I20240502 07:58:07.764110 3708 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5cc07f4a061c4d4dad410153843ce3a5: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:07.764708 3708 log.cc:824] T 00000000000000000000000000000000 P 5cc07f4a061c4d4dad410153843ce3a5: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:07.766355 3708 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cc07f4a061c4d4dad410153843ce3a5: No bootstrap required, opened a new log
I20240502 07:58:07.769136 3708 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5cc07f4a061c4d4dad410153843ce3a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cc07f4a061c4d4dad410153843ce3a5" member_type: VOTER last_known_addr { host: "127.3.139.126" port: 37175 } }
I20240502 07:58:07.769325 3708 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5cc07f4a061c4d4dad410153843ce3a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:07.769372 3708 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5cc07f4a061c4d4dad410153843ce3a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc07f4a061c4d4dad410153843ce3a5, State: Initialized, Role: FOLLOWER
I20240502 07:58:07.769775 3708 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5cc07f4a061c4d4dad410153843ce3a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cc07f4a061c4d4dad410153843ce3a5" member_type: VOTER last_known_addr { host: "127.3.139.126" port: 37175 } }
I20240502 07:58:07.769912 3708 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5cc07f4a061c4d4dad410153843ce3a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 07:58:07.769973 3708 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5cc07f4a061c4d4dad410153843ce3a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 07:58:07.770108 3708 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5cc07f4a061c4d4dad410153843ce3a5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:07.770891 3708 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5cc07f4a061c4d4dad410153843ce3a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cc07f4a061c4d4dad410153843ce3a5" member_type: VOTER last_known_addr { host: "127.3.139.126" port: 37175 } }
I20240502 07:58:07.771226 3708 leader_election.cc:302] T 00000000000000000000000000000000 P 5cc07f4a061c4d4dad410153843ce3a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5cc07f4a061c4d4dad410153843ce3a5; no voters:
I20240502 07:58:07.771494 3708 leader_election.cc:288] T 00000000000000000000000000000000 P 5cc07f4a061c4d4dad410153843ce3a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 07:58:07.771842 3713 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5cc07f4a061c4d4dad410153843ce3a5 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:07.772059 3713 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5cc07f4a061c4d4dad410153843ce3a5 [term 1 LEADER]: Becoming Leader. State: Replica: 5cc07f4a061c4d4dad410153843ce3a5, State: Running, Role: LEADER
I20240502 07:58:07.772338 3713 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5cc07f4a061c4d4dad410153843ce3a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cc07f4a061c4d4dad410153843ce3a5" member_type: VOTER last_known_addr { host: "127.3.139.126" port: 37175 } }
I20240502 07:58:07.773259 3708 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5cc07f4a061c4d4dad410153843ce3a5 [sys.catalog]: configured and running, proceeding with master startup.
I20240502 07:58:07.773525 3715 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cc07f4a061c4d4dad410153843ce3a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5cc07f4a061c4d4dad410153843ce3a5. Latest consensus state: current_term: 1 leader_uuid: "5cc07f4a061c4d4dad410153843ce3a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cc07f4a061c4d4dad410153843ce3a5" member_type: VOTER last_known_addr { host: "127.3.139.126" port: 37175 } } }
I20240502 07:58:07.773674 3715 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cc07f4a061c4d4dad410153843ce3a5 [sys.catalog]: This master's current role is: LEADER
I20240502 07:58:07.773677 3714 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cc07f4a061c4d4dad410153843ce3a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5cc07f4a061c4d4dad410153843ce3a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cc07f4a061c4d4dad410153843ce3a5" member_type: VOTER last_known_addr { host: "127.3.139.126" port: 37175 } } }
I20240502 07:58:07.773795 3714 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cc07f4a061c4d4dad410153843ce3a5 [sys.catalog]: This master's current role is: LEADER
I20240502 07:58:07.774681 3717 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240502 07:58:07.776822 3717 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240502 07:58:07.781186 3717 catalog_manager.cc:1347] Generated new cluster ID: 14640cdfdcbb4bc99fd6450a4484802d
I20240502 07:58:07.781246 3717 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240502 07:58:07.792196 3717 catalog_manager.cc:1370] Generated new certificate authority record
I20240502 07:58:07.792917 3717 catalog_manager.cc:1504] Loading token signing keys...
I20240502 07:58:07.800577 3717 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 5cc07f4a061c4d4dad410153843ce3a5: Generated new TSK 0
I20240502 07:58:07.801112 3717 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240502 07:58:07.807780 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.65:0
--local_ip_for_outbound_sockets=127.3.139.65
--webserver_interface=127.3.139.65
--webserver_port=0
--tserver_master_addrs=127.3.139.126:37175
--builtin_ntp_servers=127.3.139.84:32841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240502 07:58:07.905306 3732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:07.905486 3732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:07.905525 3732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:07.908419 3732 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:58:07.908473 3732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:07.908536 3732 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.65
I20240502 07:58:07.911944 3732 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:32841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.3.139.65
--webserver_port=0
--tserver_master_addrs=127.3.139.126:37175
--never_fsync=true
--heap_profile_path=/tmp/kudu.3732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.65
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:07.912663 3732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:07.913625 3732 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:07.921195 3738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:07.921519 3737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:07.921872 3740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:07.922302 3732 server_base.cc:1034] running on GCE node
I20240502 07:58:07.922659 3732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:07.923235 3732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:07.924407 3732 hybrid_clock.cc:648] HybridClock initialized: now 1714636687924392 us; error 33 us; skew 500 ppm
I20240502 07:58:07.935146 3732 webserver.cc:460] Webserver started at http://127.3.139.65:38591/ using document root <none> and password file <none>
I20240502 07:58:07.935631 3732 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:07.935680 3732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:07.935825 3732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:07.937259 3732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e"
format_stamp: "Formatted at 2024-05-02 07:58:07 on dist-test-slave-twcr"
I20240502 07:58:07.937657 3732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e"
format_stamp: "Formatted at 2024-05-02 07:58:07 on dist-test-slave-twcr"
I20240502 07:58:07.940538 3732 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20240502 07:58:07.942420 3746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:07.943351 3732 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:07.943440 3732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e"
format_stamp: "Formatted at 2024-05-02 07:58:07 on dist-test-slave-twcr"
I20240502 07:58:07.943517 3732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:07.957507 3732 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:07.957983 3732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:07.958112 3732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:07.958539 3732 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:07.959317 3732 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:07.959362 3732 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:07.959400 3732 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:07.959429 3732 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:07.967022 3732 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.65:35675
I20240502 07:58:07.967065 3859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.65:35675 every 8 connection(s)
I20240502 07:58:07.967783 3732 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240502 07:58:07.972321 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 3732
I20240502 07:58:07.972406 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240502 07:58:07.974536 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.66:0
--local_ip_for_outbound_sockets=127.3.139.66
--webserver_interface=127.3.139.66
--webserver_port=0
--tserver_master_addrs=127.3.139.126:37175
--builtin_ntp_servers=127.3.139.84:32841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 07:58:07.975548 3860 heartbeater.cc:347] Connected to a master server at 127.3.139.126:37175
I20240502 07:58:07.975732 3860 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:07.976156 3860 heartbeater.cc:511] Master 127.3.139.126:37175 requested a full tablet report, sending...
I20240502 07:58:07.977393 3672 ts_manager.cc:194] Registered new tserver with Master: 76cc6f9adf4c4aceac8d8f7d883deb6e (127.3.139.65:35675)
I20240502 07:58:07.978439 3672 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.65:33687
W20240502 07:58:08.071449 3863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:08.071621 3863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:08.071787 3863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:08.075670 3863 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:58:08.075722 3863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:08.075807 3863 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.66
I20240502 07:58:08.079241 3863 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:32841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.3.139.66
--webserver_port=0
--tserver_master_addrs=127.3.139.126:37175
--never_fsync=true
--heap_profile_path=/tmp/kudu.3863
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.66
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:08.079944 3863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:08.080929 3863 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:08.088084 3869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:08.088424 3868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:08.088778 3871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:08.089183 3863 server_base.cc:1034] running on GCE node
I20240502 07:58:08.089466 3863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:08.089947 3863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:08.091101 3863 hybrid_clock.cc:648] HybridClock initialized: now 1714636688091094 us; error 27 us; skew 500 ppm
I20240502 07:58:08.092621 3863 webserver.cc:460] Webserver started at http://127.3.139.66:44167/ using document root <none> and password file <none>
I20240502 07:58:08.093096 3863 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:08.093150 3863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:08.093305 3863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:08.094861 3863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "b1dd721f44bc432c87c22eb45d0345c4"
format_stamp: "Formatted at 2024-05-02 07:58:08 on dist-test-slave-twcr"
I20240502 07:58:08.095325 3863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "b1dd721f44bc432c87c22eb45d0345c4"
format_stamp: "Formatted at 2024-05-02 07:58:08 on dist-test-slave-twcr"
I20240502 07:58:08.098315 3863 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:08.100299 3877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:08.101198 3863 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:08.101291 3863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "b1dd721f44bc432c87c22eb45d0345c4"
format_stamp: "Formatted at 2024-05-02 07:58:08 on dist-test-slave-twcr"
I20240502 07:58:08.101363 3863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:08.117368 3863 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:08.117868 3863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:08.117996 3863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:08.118427 3863 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:08.119246 3863 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:08.119289 3863 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:08.119335 3863 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:08.119359 3863 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:08.126806 3863 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.66:46179
I20240502 07:58:08.126849 3990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.66:46179 every 8 connection(s)
I20240502 07:58:08.127615 3863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240502 07:58:08.129055 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 3863
I20240502 07:58:08.129158 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240502 07:58:08.131119 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.67:0
--local_ip_for_outbound_sockets=127.3.139.67
--webserver_interface=127.3.139.67
--webserver_port=0
--tserver_master_addrs=127.3.139.126:37175
--builtin_ntp_servers=127.3.139.84:32841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 07:58:08.135432 3991 heartbeater.cc:347] Connected to a master server at 127.3.139.126:37175
I20240502 07:58:08.135638 3991 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:08.136104 3991 heartbeater.cc:511] Master 127.3.139.126:37175 requested a full tablet report, sending...
I20240502 07:58:08.137020 3672 ts_manager.cc:194] Registered new tserver with Master: b1dd721f44bc432c87c22eb45d0345c4 (127.3.139.66:46179)
I20240502 07:58:08.137528 3672 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.66:47123
W20240502 07:58:08.229451 3994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:08.229625 3994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:08.229678 3994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:08.232563 3994 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:58:08.232618 3994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:08.232698 3994 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.67
I20240502 07:58:08.236122 3994 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:32841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.3.139.67
--webserver_port=0
--tserver_master_addrs=127.3.139.126:37175
--never_fsync=true
--heap_profile_path=/tmp/kudu.3994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.67
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:08.236816 3994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:08.237752 3994 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:08.244781 3999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:08.245131 4002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:08.245461 4000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:08.245914 3994 server_base.cc:1034] running on GCE node
I20240502 07:58:08.246239 3994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:08.246724 3994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:08.247867 3994 hybrid_clock.cc:648] HybridClock initialized: now 1714636688247848 us; error 30 us; skew 500 ppm
I20240502 07:58:08.249428 3994 webserver.cc:460] Webserver started at http://127.3.139.67:33843/ using document root <none> and password file <none>
I20240502 07:58:08.249899 3994 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:08.249953 3994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:08.250106 3994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:08.251592 3994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "0751e317d08443a18b12c780308f9c12"
format_stamp: "Formatted at 2024-05-02 07:58:08 on dist-test-slave-twcr"
I20240502 07:58:08.252018 3994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "0751e317d08443a18b12c780308f9c12"
format_stamp: "Formatted at 2024-05-02 07:58:08 on dist-test-slave-twcr"
I20240502 07:58:08.254874 3994 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240502 07:58:08.256780 4008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:08.257637 3994 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:08.257730 3994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "0751e317d08443a18b12c780308f9c12"
format_stamp: "Formatted at 2024-05-02 07:58:08 on dist-test-slave-twcr"
I20240502 07:58:08.257802 3994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:08.274768 3994 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:08.275328 3994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:08.275465 3994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:08.275914 3994 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:08.276684 3994 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:08.276729 3994 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:08.276775 3994 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:08.276803 3994 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:08.284284 3994 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.67:33895
I20240502 07:58:08.284320 4121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.67:33895 every 8 connection(s)
I20240502 07:58:08.285085 3994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240502 07:58:08.286365 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 3994
I20240502 07:58:08.286455 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240502 07:58:08.292850 4122 heartbeater.cc:347] Connected to a master server at 127.3.139.126:37175
I20240502 07:58:08.293020 4122 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:08.293445 4122 heartbeater.cc:511] Master 127.3.139.126:37175 requested a full tablet report, sending...
I20240502 07:58:08.294276 3672 ts_manager.cc:194] Registered new tserver with Master: 0751e317d08443a18b12c780308f9c12 (127.3.139.67:33895)
I20240502 07:58:08.294832 3672 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.67:40297
I20240502 07:58:08.300349 3629 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20240502 07:58:08.310211 3629 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20240502 07:58:08.980262 3860 heartbeater.cc:503] Master 127.3.139.126:37175 was elected leader, sending a full tablet report...
I20240502 07:58:09.139209 3991 heartbeater.cc:503] Master 127.3.139.126:37175 was elected leader, sending a full tablet report...
I20240502 07:58:09.296485 4122 heartbeater.cc:503] Master 127.3.139.126:37175 was elected leader, sending a full tablet report...
I20240502 07:58:12.062700 3672 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49232:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\200\007\001\000\000\000\000\000\006\001\000\000\000\000\000\007\001\000\377\377\377\177"
indirect_data: "<redacted>"""
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 8
seed: 0
}
}
W20240502 07:58:12.065691 3672 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240502 07:58:12.087599 3917 tablet_service.cc:1467] Processing CreateTablet for tablet c501775259824616959fab8bca85a93b (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240502 07:58:12.087599 3792 tablet_service.cc:1467] Processing CreateTablet for tablet 87abd4c4107e4a4aa7dfdb55ee00b26f (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240502 07:58:12.087941 3915 tablet_service.cc:1467] Processing CreateTablet for tablet 1839e64af65e4171a1222704f19d1856 (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240502 07:58:12.088127 3922 tablet_service.cc:1467] Processing CreateTablet for tablet e7049163a60b47c9899ffc9ad679df95 (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 07:58:12.088595 3917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c501775259824616959fab8bca85a93b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.088595 3792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87abd4c4107e4a4aa7dfdb55ee00b26f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.088920 3922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7049163a60b47c9899ffc9ad679df95. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.089682 3915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1839e64af65e4171a1222704f19d1856. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.091305 3923 tablet_service.cc:1467] Processing CreateTablet for tablet 87abd4c4107e4a4aa7dfdb55ee00b26f (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240502 07:58:12.091552 3923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87abd4c4107e4a4aa7dfdb55ee00b26f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.091596 3916 tablet_service.cc:1467] Processing CreateTablet for tablet 94701214c7954905874450dcfa9a678b (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 07:58:12.091790 3916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94701214c7954905874450dcfa9a678b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.093454 3914 tablet_service.cc:1467] Processing CreateTablet for tablet 86519f378c294fca8232a94970dd5339 (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 07:58:12.093659 3914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86519f378c294fca8232a94970dd5339. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.093822 4043 tablet_service.cc:1467] Processing CreateTablet for tablet 45fa74d198214dc78d63ce787c1490d6 (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 07:58:12.094410 4054 tablet_service.cc:1467] Processing CreateTablet for tablet 87abd4c4107e4a4aa7dfdb55ee00b26f (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240502 07:58:12.094671 4043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45fa74d198214dc78d63ce787c1490d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.094799 4053 tablet_service.cc:1467] Processing CreateTablet for tablet e7049163a60b47c9899ffc9ad679df95 (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 07:58:12.094779 3791 tablet_service.cc:1467] Processing CreateTablet for tablet e7049163a60b47c9899ffc9ad679df95 (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 07:58:12.095016 4053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7049163a60b47c9899ffc9ad679df95. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.095064 3791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7049163a60b47c9899ffc9ad679df95. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.095456 3913 tablet_service.cc:1467] Processing CreateTablet for tablet c7f79b16b7f949f0aa891ef70bdf6608 (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240502 07:58:12.095675 3913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7f79b16b7f949f0aa891ef70bdf6608. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.095723 4052 tablet_service.cc:1467] Processing CreateTablet for tablet 2a08d03748e94ebaab74067281c8f8a2 (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240502 07:58:12.095968 4052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a08d03748e94ebaab74067281c8f8a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.096462 4051 tablet_service.cc:1467] Processing CreateTablet for tablet 02538cc46cb6465d927569ed749c2154 (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 07:58:12.096701 4051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02538cc46cb6465d927569ed749c2154. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.097467 3912 tablet_service.cc:1467] Processing CreateTablet for tablet 45fa74d198214dc78d63ce787c1490d6 (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 07:58:12.097677 3912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45fa74d198214dc78d63ce787c1490d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.094111 4055 tablet_service.cc:1467] Processing CreateTablet for tablet 23d6b429a6374ccd9e5e529186fdcea9 (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 07:58:12.097929 4055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23d6b429a6374ccd9e5e529186fdcea9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.099144 3921 tablet_service.cc:1467] Processing CreateTablet for tablet 2a08d03748e94ebaab74067281c8f8a2 (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240502 07:58:12.099238 3911 tablet_service.cc:1467] Processing CreateTablet for tablet a3bb326555d747479c66254d5c592b57 (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240502 07:58:12.099272 4050 tablet_service.cc:1467] Processing CreateTablet for tablet e2bf12199a744839a7e40531e53d3aea (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240502 07:58:12.099352 3921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a08d03748e94ebaab74067281c8f8a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.099505 4050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2bf12199a744839a7e40531e53d3aea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.099589 3910 tablet_service.cc:1467] Processing CreateTablet for tablet 9b5aa286c46445a381e581dff1798ed8 (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 07:58:12.099802 3910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b5aa286c46445a381e581dff1798ed8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.093582 4045 tablet_service.cc:1467] Processing CreateTablet for tablet 86519f378c294fca8232a94970dd5339 (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 07:58:12.100893 4045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86519f378c294fca8232a94970dd5339. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.101130 3790 tablet_service.cc:1467] Processing CreateTablet for tablet 2a08d03748e94ebaab74067281c8f8a2 (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240502 07:58:12.101356 3790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a08d03748e94ebaab74067281c8f8a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.101353 4049 tablet_service.cc:1467] Processing CreateTablet for tablet b99f9c244e014973a024cb91904a64f8 (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 07:58:12.101590 4049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b99f9c244e014973a024cb91904a64f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.101953 3925 tablet_service.cc:1467] Processing CreateTablet for tablet 11e2dc546eb3466fab164bfcb0bac310 (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240502 07:58:12.102169 3925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11e2dc546eb3466fab164bfcb0bac310. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.087939 3919 tablet_service.cc:1467] Processing CreateTablet for tablet e2bf12199a744839a7e40531e53d3aea (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240502 07:58:12.103225 3919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2bf12199a744839a7e40531e53d3aea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.103312 3789 tablet_service.cc:1467] Processing CreateTablet for tablet 02538cc46cb6465d927569ed749c2154 (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 07:58:12.103435 4048 tablet_service.cc:1467] Processing CreateTablet for tablet c501775259824616959fab8bca85a93b (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240502 07:58:12.103516 3789 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02538cc46cb6465d927569ed749c2154. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.103652 4048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c501775259824616959fab8bca85a93b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.104923 3911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3bb326555d747479c66254d5c592b57. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.105321 3788 tablet_service.cc:1467] Processing CreateTablet for tablet e2bf12199a744839a7e40531e53d3aea (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240502 07:58:12.105449 4047 tablet_service.cc:1467] Processing CreateTablet for tablet 94701214c7954905874450dcfa9a678b (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 07:58:12.105543 3788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2bf12199a744839a7e40531e53d3aea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.105664 4047 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94701214c7954905874450dcfa9a678b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.087390 3918 tablet_service.cc:1467] Processing CreateTablet for tablet b99f9c244e014973a024cb91904a64f8 (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 07:58:12.106810 3918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b99f9c244e014973a024cb91904a64f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.107318 4142 tablet_bootstrap.cc:492] T 23d6b429a6374ccd9e5e529186fdcea9 P 0751e317d08443a18b12c780308f9c12: Bootstrap starting.
I20240502 07:58:12.107447 4046 tablet_service.cc:1467] Processing CreateTablet for tablet 1839e64af65e4171a1222704f19d1856 (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240502 07:58:12.107652 4046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1839e64af65e4171a1222704f19d1856. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.107761 3787 tablet_service.cc:1467] Processing CreateTablet for tablet b99f9c244e014973a024cb91904a64f8 (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 07:58:12.107985 3787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b99f9c244e014973a024cb91904a64f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.108908 4140 tablet_bootstrap.cc:492] T 87abd4c4107e4a4aa7dfdb55ee00b26f P b1dd721f44bc432c87c22eb45d0345c4: Bootstrap starting.
I20240502 07:58:12.109426 4044 tablet_service.cc:1467] Processing CreateTablet for tablet c7f79b16b7f949f0aa891ef70bdf6608 (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240502 07:58:12.109647 4044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7f79b16b7f949f0aa891ef70bdf6608. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.109786 3786 tablet_service.cc:1467] Processing CreateTablet for tablet c501775259824616959fab8bca85a93b (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240502 07:58:12.087388 3920 tablet_service.cc:1467] Processing CreateTablet for tablet 02538cc46cb6465d927569ed749c2154 (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 07:58:12.110013 3920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02538cc46cb6465d927569ed749c2154. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.110018 3786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c501775259824616959fab8bca85a93b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.111217 4140 tablet_bootstrap.cc:654] T 87abd4c4107e4a4aa7dfdb55ee00b26f P b1dd721f44bc432c87c22eb45d0345c4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.111479 4042 tablet_service.cc:1467] Processing CreateTablet for tablet a3bb326555d747479c66254d5c592b57 (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240502 07:58:12.111688 4042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3bb326555d747479c66254d5c592b57. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.111848 3785 tablet_service.cc:1467] Processing CreateTablet for tablet 94701214c7954905874450dcfa9a678b (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 07:58:12.111954 4140 log.cc:824] T 87abd4c4107e4a4aa7dfdb55ee00b26f P b1dd721f44bc432c87c22eb45d0345c4: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:12.112084 3785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94701214c7954905874450dcfa9a678b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.113185 4141 tablet_bootstrap.cc:492] T e7049163a60b47c9899ffc9ad679df95 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Bootstrap starting.
I20240502 07:58:12.113479 4041 tablet_service.cc:1467] Processing CreateTablet for tablet 9b5aa286c46445a381e581dff1798ed8 (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 07:58:12.113581 4142 tablet_bootstrap.cc:654] T 23d6b429a6374ccd9e5e529186fdcea9 P 0751e317d08443a18b12c780308f9c12: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.113689 4041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b5aa286c46445a381e581dff1798ed8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.113888 3784 tablet_service.cc:1467] Processing CreateTablet for tablet 1839e64af65e4171a1222704f19d1856 (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240502 07:58:12.114104 3784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1839e64af65e4171a1222704f19d1856. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.114887 4141 tablet_bootstrap.cc:654] T e7049163a60b47c9899ffc9ad679df95 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.114621 4142 log.cc:824] T 23d6b429a6374ccd9e5e529186fdcea9 P 0751e317d08443a18b12c780308f9c12: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:12.115574 4141 log.cc:824] T e7049163a60b47c9899ffc9ad679df95 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:12.115967 3783 tablet_service.cc:1467] Processing CreateTablet for tablet 86519f378c294fca8232a94970dd5339 (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 07:58:12.116086 4054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87abd4c4107e4a4aa7dfdb55ee00b26f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.116194 3783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86519f378c294fca8232a94970dd5339. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.116904 4142 tablet_bootstrap.cc:492] T 23d6b429a6374ccd9e5e529186fdcea9 P 0751e317d08443a18b12c780308f9c12: No bootstrap required, opened a new log
I20240502 07:58:12.117050 4142 ts_tablet_manager.cc:1367] T 23d6b429a6374ccd9e5e529186fdcea9 P 0751e317d08443a18b12c780308f9c12: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20240502 07:58:12.117276 4141 tablet_bootstrap.cc:492] T e7049163a60b47c9899ffc9ad679df95 P 76cc6f9adf4c4aceac8d8f7d883deb6e: No bootstrap required, opened a new log
I20240502 07:58:12.117426 4141 ts_tablet_manager.cc:1367] T e7049163a60b47c9899ffc9ad679df95 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240502 07:58:12.117983 3782 tablet_service.cc:1467] Processing CreateTablet for tablet c7f79b16b7f949f0aa891ef70bdf6608 (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240502 07:58:12.118197 3782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7f79b16b7f949f0aa891ef70bdf6608. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.119328 3781 tablet_service.cc:1467] Processing CreateTablet for tablet 45fa74d198214dc78d63ce787c1490d6 (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 07:58:12.119575 3781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45fa74d198214dc78d63ce787c1490d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.119931 4142 raft_consensus.cc:348] T 23d6b429a6374ccd9e5e529186fdcea9 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.120107 4142 raft_consensus.cc:374] T 23d6b429a6374ccd9e5e529186fdcea9 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.120157 4142 raft_consensus.cc:729] T 23d6b429a6374ccd9e5e529186fdcea9 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0751e317d08443a18b12c780308f9c12, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.120397 4141 raft_consensus.cc:348] T e7049163a60b47c9899ffc9ad679df95 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.120568 4141 raft_consensus.cc:374] T e7049163a60b47c9899ffc9ad679df95 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.120616 4141 raft_consensus.cc:729] T e7049163a60b47c9899ffc9ad679df95 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76cc6f9adf4c4aceac8d8f7d883deb6e, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.093582 4056 tablet_service.cc:1467] Processing CreateTablet for tablet 11e2dc546eb3466fab164bfcb0bac310 (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240502 07:58:12.121063 4141 consensus_queue.cc:260] T e7049163a60b47c9899ffc9ad679df95 P 76cc6f9adf4c4aceac8d8f7d883deb6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.121263 4056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11e2dc546eb3466fab164bfcb0bac310. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.121281 4140 tablet_bootstrap.cc:492] T 87abd4c4107e4a4aa7dfdb55ee00b26f P b1dd721f44bc432c87c22eb45d0345c4: No bootstrap required, opened a new log
I20240502 07:58:12.121410 4140 ts_tablet_manager.cc:1367] T 87abd4c4107e4a4aa7dfdb55ee00b26f P b1dd721f44bc432c87c22eb45d0345c4: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.000s
I20240502 07:58:12.121433 3780 tablet_service.cc:1467] Processing CreateTablet for tablet a3bb326555d747479c66254d5c592b57 (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240502 07:58:12.121644 3780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3bb326555d747479c66254d5c592b57. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.121845 4141 ts_tablet_manager.cc:1398] T e7049163a60b47c9899ffc9ad679df95 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20240502 07:58:12.123245 4141 tablet_bootstrap.cc:492] T 2a08d03748e94ebaab74067281c8f8a2 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Bootstrap starting.
I20240502 07:58:12.123437 3779 tablet_service.cc:1467] Processing CreateTablet for tablet 9b5aa286c46445a381e581dff1798ed8 (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 07:58:12.123656 3779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b5aa286c46445a381e581dff1798ed8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.124001 4141 tablet_bootstrap.cc:654] T 2a08d03748e94ebaab74067281c8f8a2 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.124181 4140 raft_consensus.cc:348] T 87abd4c4107e4a4aa7dfdb55ee00b26f P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.124315 4140 raft_consensus.cc:374] T 87abd4c4107e4a4aa7dfdb55ee00b26f P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.124349 4140 raft_consensus.cc:729] T 87abd4c4107e4a4aa7dfdb55ee00b26f P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1dd721f44bc432c87c22eb45d0345c4, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.125020 4141 tablet_bootstrap.cc:492] T 2a08d03748e94ebaab74067281c8f8a2 P 76cc6f9adf4c4aceac8d8f7d883deb6e: No bootstrap required, opened a new log
I20240502 07:58:12.125088 4141 ts_tablet_manager.cc:1367] T 2a08d03748e94ebaab74067281c8f8a2 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:12.125545 4141 raft_consensus.cc:348] T 2a08d03748e94ebaab74067281c8f8a2 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.125633 4141 raft_consensus.cc:374] T 2a08d03748e94ebaab74067281c8f8a2 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.125661 4141 raft_consensus.cc:729] T 2a08d03748e94ebaab74067281c8f8a2 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76cc6f9adf4c4aceac8d8f7d883deb6e, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.125622 3924 tablet_service.cc:1467] Processing CreateTablet for tablet 23d6b429a6374ccd9e5e529186fdcea9 (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 07:58:12.125814 3924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23d6b429a6374ccd9e5e529186fdcea9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.125772 4141 consensus_queue.cc:260] T 2a08d03748e94ebaab74067281c8f8a2 P 76cc6f9adf4c4aceac8d8f7d883deb6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.126097 4141 ts_tablet_manager.cc:1398] T 2a08d03748e94ebaab74067281c8f8a2 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:12.126242 4141 tablet_bootstrap.cc:492] T 02538cc46cb6465d927569ed749c2154 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Bootstrap starting.
I20240502 07:58:12.127084 4141 tablet_bootstrap.cc:654] T 02538cc46cb6465d927569ed749c2154 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.087395 3793 tablet_service.cc:1467] Processing CreateTablet for tablet 23d6b429a6374ccd9e5e529186fdcea9 (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 07:58:12.127701 3793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23d6b429a6374ccd9e5e529186fdcea9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.128582 4140 consensus_queue.cc:260] T 87abd4c4107e4a4aa7dfdb55ee00b26f P b1dd721f44bc432c87c22eb45d0345c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.129287 4140 ts_tablet_manager.cc:1398] T 87abd4c4107e4a4aa7dfdb55ee00b26f P b1dd721f44bc432c87c22eb45d0345c4: Time spent starting tablet: real 0.008s user 0.004s sys 0.000s
I20240502 07:58:12.129626 4142 consensus_queue.cc:260] T 23d6b429a6374ccd9e5e529186fdcea9 P 0751e317d08443a18b12c780308f9c12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.087395 3794 tablet_service.cc:1467] Processing CreateTablet for tablet 11e2dc546eb3466fab164bfcb0bac310 (DEFAULT_TABLE table=rebalance_test_table_0 [id=1317b8ddec864b87ae8b398d1ff59426]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION -2147483648 <= VALUES < 0
W20240502 07:58:12.130729 3992 tablet.cc:2369] T 87abd4c4107e4a4aa7dfdb55ee00b26f P b1dd721f44bc432c87c22eb45d0345c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 07:58:12.130992 3794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11e2dc546eb3466fab164bfcb0bac310. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.133185 4141 tablet_bootstrap.cc:492] T 02538cc46cb6465d927569ed749c2154 P 76cc6f9adf4c4aceac8d8f7d883deb6e: No bootstrap required, opened a new log
I20240502 07:58:12.133262 4141 ts_tablet_manager.cc:1367] T 02538cc46cb6465d927569ed749c2154 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 07:58:12.133438 4142 ts_tablet_manager.cc:1398] T 23d6b429a6374ccd9e5e529186fdcea9 P 0751e317d08443a18b12c780308f9c12: Time spent starting tablet: real 0.016s user 0.005s sys 0.000s
I20240502 07:58:12.133562 4140 tablet_bootstrap.cc:492] T 86519f378c294fca8232a94970dd5339 P b1dd721f44bc432c87c22eb45d0345c4: Bootstrap starting.
I20240502 07:58:12.133630 4141 raft_consensus.cc:348] T 02538cc46cb6465d927569ed749c2154 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.133714 4141 raft_consensus.cc:374] T 02538cc46cb6465d927569ed749c2154 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.133744 4141 raft_consensus.cc:729] T 02538cc46cb6465d927569ed749c2154 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76cc6f9adf4c4aceac8d8f7d883deb6e, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.133846 4141 consensus_queue.cc:260] T 02538cc46cb6465d927569ed749c2154 P 76cc6f9adf4c4aceac8d8f7d883deb6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.134100 4141 ts_tablet_manager.cc:1398] T 02538cc46cb6465d927569ed749c2154 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:12.134243 4141 tablet_bootstrap.cc:492] T e2bf12199a744839a7e40531e53d3aea P 76cc6f9adf4c4aceac8d8f7d883deb6e: Bootstrap starting.
I20240502 07:58:12.134982 4141 tablet_bootstrap.cc:654] T e2bf12199a744839a7e40531e53d3aea P 76cc6f9adf4c4aceac8d8f7d883deb6e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.135649 4142 tablet_bootstrap.cc:492] T e2bf12199a744839a7e40531e53d3aea P 0751e317d08443a18b12c780308f9c12: Bootstrap starting.
I20240502 07:58:12.136121 4141 tablet_bootstrap.cc:492] T e2bf12199a744839a7e40531e53d3aea P 76cc6f9adf4c4aceac8d8f7d883deb6e: No bootstrap required, opened a new log
I20240502 07:58:12.136199 4141 ts_tablet_manager.cc:1367] T e2bf12199a744839a7e40531e53d3aea P 76cc6f9adf4c4aceac8d8f7d883deb6e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:12.136479 4142 tablet_bootstrap.cc:654] T e2bf12199a744839a7e40531e53d3aea P 0751e317d08443a18b12c780308f9c12: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.136576 4141 raft_consensus.cc:348] T e2bf12199a744839a7e40531e53d3aea P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.136660 4141 raft_consensus.cc:374] T e2bf12199a744839a7e40531e53d3aea P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.136687 4141 raft_consensus.cc:729] T e2bf12199a744839a7e40531e53d3aea P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76cc6f9adf4c4aceac8d8f7d883deb6e, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.136801 4141 consensus_queue.cc:260] T e2bf12199a744839a7e40531e53d3aea P 76cc6f9adf4c4aceac8d8f7d883deb6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.137045 4141 ts_tablet_manager.cc:1398] T e2bf12199a744839a7e40531e53d3aea P 76cc6f9adf4c4aceac8d8f7d883deb6e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:12.137310 4141 tablet_bootstrap.cc:492] T b99f9c244e014973a024cb91904a64f8 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Bootstrap starting.
I20240502 07:58:12.137323 4140 tablet_bootstrap.cc:654] T 86519f378c294fca8232a94970dd5339 P b1dd721f44bc432c87c22eb45d0345c4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.138110 4141 tablet_bootstrap.cc:654] T b99f9c244e014973a024cb91904a64f8 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.138725 4142 tablet_bootstrap.cc:492] T e2bf12199a744839a7e40531e53d3aea P 0751e317d08443a18b12c780308f9c12: No bootstrap required, opened a new log
I20240502 07:58:12.138809 4142 ts_tablet_manager.cc:1367] T e2bf12199a744839a7e40531e53d3aea P 0751e317d08443a18b12c780308f9c12: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:58:12.139292 4141 tablet_bootstrap.cc:492] T b99f9c244e014973a024cb91904a64f8 P 76cc6f9adf4c4aceac8d8f7d883deb6e: No bootstrap required, opened a new log
I20240502 07:58:12.139379 4141 ts_tablet_manager.cc:1367] T b99f9c244e014973a024cb91904a64f8 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:12.139328 4142 raft_consensus.cc:348] T e2bf12199a744839a7e40531e53d3aea P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.139420 4142 raft_consensus.cc:374] T e2bf12199a744839a7e40531e53d3aea P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.139449 4142 raft_consensus.cc:729] T e2bf12199a744839a7e40531e53d3aea P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0751e317d08443a18b12c780308f9c12, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.139564 4142 consensus_queue.cc:260] T e2bf12199a744839a7e40531e53d3aea P 0751e317d08443a18b12c780308f9c12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.139801 4142 ts_tablet_manager.cc:1398] T e2bf12199a744839a7e40531e53d3aea P 0751e317d08443a18b12c780308f9c12: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:12.139840 4141 raft_consensus.cc:348] T b99f9c244e014973a024cb91904a64f8 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.139935 4141 raft_consensus.cc:374] T b99f9c244e014973a024cb91904a64f8 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.139966 4141 raft_consensus.cc:729] T b99f9c244e014973a024cb91904a64f8 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76cc6f9adf4c4aceac8d8f7d883deb6e, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.140020 4142 tablet_bootstrap.cc:492] T 86519f378c294fca8232a94970dd5339 P 0751e317d08443a18b12c780308f9c12: Bootstrap starting.
I20240502 07:58:12.140092 4141 consensus_queue.cc:260] T b99f9c244e014973a024cb91904a64f8 P 76cc6f9adf4c4aceac8d8f7d883deb6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.140415 4140 tablet_bootstrap.cc:492] T 86519f378c294fca8232a94970dd5339 P b1dd721f44bc432c87c22eb45d0345c4: No bootstrap required, opened a new log
I20240502 07:58:12.140484 4140 ts_tablet_manager.cc:1367] T 86519f378c294fca8232a94970dd5339 P b1dd721f44bc432c87c22eb45d0345c4: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 07:58:12.140880 4142 tablet_bootstrap.cc:654] T 86519f378c294fca8232a94970dd5339 P 0751e317d08443a18b12c780308f9c12: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.140889 4141 ts_tablet_manager.cc:1398] T b99f9c244e014973a024cb91904a64f8 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:12.140974 4140 raft_consensus.cc:348] T 86519f378c294fca8232a94970dd5339 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.141037 4141 tablet_bootstrap.cc:492] T c501775259824616959fab8bca85a93b P 76cc6f9adf4c4aceac8d8f7d883deb6e: Bootstrap starting.
I20240502 07:58:12.141057 4140 raft_consensus.cc:374] T 86519f378c294fca8232a94970dd5339 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.141085 4140 raft_consensus.cc:729] T 86519f378c294fca8232a94970dd5339 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1dd721f44bc432c87c22eb45d0345c4, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.141192 4140 consensus_queue.cc:260] T 86519f378c294fca8232a94970dd5339 P b1dd721f44bc432c87c22eb45d0345c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.141424 4140 ts_tablet_manager.cc:1398] T 86519f378c294fca8232a94970dd5339 P b1dd721f44bc432c87c22eb45d0345c4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:12.141570 4140 tablet_bootstrap.cc:492] T c7f79b16b7f949f0aa891ef70bdf6608 P b1dd721f44bc432c87c22eb45d0345c4: Bootstrap starting.
I20240502 07:58:12.141822 4141 tablet_bootstrap.cc:654] T c501775259824616959fab8bca85a93b P 76cc6f9adf4c4aceac8d8f7d883deb6e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.142326 4140 tablet_bootstrap.cc:654] T c7f79b16b7f949f0aa891ef70bdf6608 P b1dd721f44bc432c87c22eb45d0345c4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.143256 4140 tablet_bootstrap.cc:492] T c7f79b16b7f949f0aa891ef70bdf6608 P b1dd721f44bc432c87c22eb45d0345c4: No bootstrap required, opened a new log
I20240502 07:58:12.143342 4140 ts_tablet_manager.cc:1367] T c7f79b16b7f949f0aa891ef70bdf6608 P b1dd721f44bc432c87c22eb45d0345c4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:12.143664 4140 raft_consensus.cc:348] T c7f79b16b7f949f0aa891ef70bdf6608 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.143738 4140 raft_consensus.cc:374] T c7f79b16b7f949f0aa891ef70bdf6608 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.143766 4140 raft_consensus.cc:729] T c7f79b16b7f949f0aa891ef70bdf6608 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1dd721f44bc432c87c22eb45d0345c4, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.143895 4140 consensus_queue.cc:260] T c7f79b16b7f949f0aa891ef70bdf6608 P b1dd721f44bc432c87c22eb45d0345c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.144043 4141 tablet_bootstrap.cc:492] T c501775259824616959fab8bca85a93b P 76cc6f9adf4c4aceac8d8f7d883deb6e: No bootstrap required, opened a new log
I20240502 07:58:12.144109 4141 ts_tablet_manager.cc:1367] T c501775259824616959fab8bca85a93b P 76cc6f9adf4c4aceac8d8f7d883deb6e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:58:12.144138 4140 ts_tablet_manager.cc:1398] T c7f79b16b7f949f0aa891ef70bdf6608 P b1dd721f44bc432c87c22eb45d0345c4: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:12.144243 4142 tablet_bootstrap.cc:492] T 86519f378c294fca8232a94970dd5339 P 0751e317d08443a18b12c780308f9c12: No bootstrap required, opened a new log
I20240502 07:58:12.144275 4140 tablet_bootstrap.cc:492] T 2a08d03748e94ebaab74067281c8f8a2 P b1dd721f44bc432c87c22eb45d0345c4: Bootstrap starting.
I20240502 07:58:12.144311 4142 ts_tablet_manager.cc:1367] T 86519f378c294fca8232a94970dd5339 P 0751e317d08443a18b12c780308f9c12: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 07:58:12.144583 4141 raft_consensus.cc:348] T c501775259824616959fab8bca85a93b P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.144673 4141 raft_consensus.cc:374] T c501775259824616959fab8bca85a93b P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.144702 4141 raft_consensus.cc:729] T c501775259824616959fab8bca85a93b P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76cc6f9adf4c4aceac8d8f7d883deb6e, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.144707 4142 raft_consensus.cc:348] T 86519f378c294fca8232a94970dd5339 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.144791 4142 raft_consensus.cc:374] T 86519f378c294fca8232a94970dd5339 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.144821 4142 raft_consensus.cc:729] T 86519f378c294fca8232a94970dd5339 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0751e317d08443a18b12c780308f9c12, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.144836 4141 consensus_queue.cc:260] T c501775259824616959fab8bca85a93b P 76cc6f9adf4c4aceac8d8f7d883deb6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.144920 4142 consensus_queue.cc:260] T 86519f378c294fca8232a94970dd5339 P 0751e317d08443a18b12c780308f9c12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.145083 4140 tablet_bootstrap.cc:654] T 2a08d03748e94ebaab74067281c8f8a2 P b1dd721f44bc432c87c22eb45d0345c4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.145144 4141 ts_tablet_manager.cc:1398] T c501775259824616959fab8bca85a93b P 76cc6f9adf4c4aceac8d8f7d883deb6e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:12.145164 4142 ts_tablet_manager.cc:1398] T 86519f378c294fca8232a94970dd5339 P 0751e317d08443a18b12c780308f9c12: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:12.145287 4141 tablet_bootstrap.cc:492] T 94701214c7954905874450dcfa9a678b P 76cc6f9adf4c4aceac8d8f7d883deb6e: Bootstrap starting.
I20240502 07:58:12.145309 4142 tablet_bootstrap.cc:492] T b99f9c244e014973a024cb91904a64f8 P 0751e317d08443a18b12c780308f9c12: Bootstrap starting.
I20240502 07:58:12.146054 4141 tablet_bootstrap.cc:654] T 94701214c7954905874450dcfa9a678b P 76cc6f9adf4c4aceac8d8f7d883deb6e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.146090 4142 tablet_bootstrap.cc:654] T b99f9c244e014973a024cb91904a64f8 P 0751e317d08443a18b12c780308f9c12: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.147066 4141 tablet_bootstrap.cc:492] T 94701214c7954905874450dcfa9a678b P 76cc6f9adf4c4aceac8d8f7d883deb6e: No bootstrap required, opened a new log
I20240502 07:58:12.147135 4141 ts_tablet_manager.cc:1367] T 94701214c7954905874450dcfa9a678b P 76cc6f9adf4c4aceac8d8f7d883deb6e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:12.147487 4141 raft_consensus.cc:348] T 94701214c7954905874450dcfa9a678b P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.147576 4141 raft_consensus.cc:374] T 94701214c7954905874450dcfa9a678b P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.147607 4141 raft_consensus.cc:729] T 94701214c7954905874450dcfa9a678b P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76cc6f9adf4c4aceac8d8f7d883deb6e, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.147701 4141 consensus_queue.cc:260] T 94701214c7954905874450dcfa9a678b P 76cc6f9adf4c4aceac8d8f7d883deb6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.147938 4141 ts_tablet_manager.cc:1398] T 94701214c7954905874450dcfa9a678b P 76cc6f9adf4c4aceac8d8f7d883deb6e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:12.148089 4141 tablet_bootstrap.cc:492] T 1839e64af65e4171a1222704f19d1856 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Bootstrap starting.
I20240502 07:58:12.148756 4141 tablet_bootstrap.cc:654] T 1839e64af65e4171a1222704f19d1856 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.149200 4142 tablet_bootstrap.cc:492] T b99f9c244e014973a024cb91904a64f8 P 0751e317d08443a18b12c780308f9c12: No bootstrap required, opened a new log
I20240502 07:58:12.149277 4142 ts_tablet_manager.cc:1367] T b99f9c244e014973a024cb91904a64f8 P 0751e317d08443a18b12c780308f9c12: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 07:58:12.149551 4140 tablet_bootstrap.cc:492] T 2a08d03748e94ebaab74067281c8f8a2 P b1dd721f44bc432c87c22eb45d0345c4: No bootstrap required, opened a new log
I20240502 07:58:12.149603 4140 ts_tablet_manager.cc:1367] T 2a08d03748e94ebaab74067281c8f8a2 P b1dd721f44bc432c87c22eb45d0345c4: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20240502 07:58:12.149667 4142 raft_consensus.cc:348] T b99f9c244e014973a024cb91904a64f8 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.149756 4142 raft_consensus.cc:374] T b99f9c244e014973a024cb91904a64f8 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.149788 4142 raft_consensus.cc:729] T b99f9c244e014973a024cb91904a64f8 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0751e317d08443a18b12c780308f9c12, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.149894 4142 consensus_queue.cc:260] T b99f9c244e014973a024cb91904a64f8 P 0751e317d08443a18b12c780308f9c12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.149895 4140 raft_consensus.cc:348] T 2a08d03748e94ebaab74067281c8f8a2 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.149976 4140 raft_consensus.cc:374] T 2a08d03748e94ebaab74067281c8f8a2 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.150004 4140 raft_consensus.cc:729] T 2a08d03748e94ebaab74067281c8f8a2 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1dd721f44bc432c87c22eb45d0345c4, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.150116 4142 ts_tablet_manager.cc:1398] T b99f9c244e014973a024cb91904a64f8 P 0751e317d08443a18b12c780308f9c12: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:12.150100 4140 consensus_queue.cc:260] T 2a08d03748e94ebaab74067281c8f8a2 P b1dd721f44bc432c87c22eb45d0345c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.150265 4142 tablet_bootstrap.cc:492] T 2a08d03748e94ebaab74067281c8f8a2 P 0751e317d08443a18b12c780308f9c12: Bootstrap starting.
I20240502 07:58:12.150313 4140 ts_tablet_manager.cc:1398] T 2a08d03748e94ebaab74067281c8f8a2 P b1dd721f44bc432c87c22eb45d0345c4: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:12.150453 4140 tablet_bootstrap.cc:492] T 9b5aa286c46445a381e581dff1798ed8 P b1dd721f44bc432c87c22eb45d0345c4: Bootstrap starting.
I20240502 07:58:12.151072 4142 tablet_bootstrap.cc:654] T 2a08d03748e94ebaab74067281c8f8a2 P 0751e317d08443a18b12c780308f9c12: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.151242 4140 tablet_bootstrap.cc:654] T 9b5aa286c46445a381e581dff1798ed8 P b1dd721f44bc432c87c22eb45d0345c4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.152050 4142 tablet_bootstrap.cc:492] T 2a08d03748e94ebaab74067281c8f8a2 P 0751e317d08443a18b12c780308f9c12: No bootstrap required, opened a new log
I20240502 07:58:12.152171 4142 ts_tablet_manager.cc:1367] T 2a08d03748e94ebaab74067281c8f8a2 P 0751e317d08443a18b12c780308f9c12: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:12.152227 4140 tablet_bootstrap.cc:492] T 9b5aa286c46445a381e581dff1798ed8 P b1dd721f44bc432c87c22eb45d0345c4: No bootstrap required, opened a new log
I20240502 07:58:12.152302 4140 ts_tablet_manager.cc:1367] T 9b5aa286c46445a381e581dff1798ed8 P b1dd721f44bc432c87c22eb45d0345c4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240502 07:58:12.152576 4141 tablet_bootstrap.cc:492] T 1839e64af65e4171a1222704f19d1856 P 76cc6f9adf4c4aceac8d8f7d883deb6e: No bootstrap required, opened a new log
I20240502 07:58:12.152649 4141 ts_tablet_manager.cc:1367] T 1839e64af65e4171a1222704f19d1856 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 07:58:12.152702 4142 raft_consensus.cc:348] T 2a08d03748e94ebaab74067281c8f8a2 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.152698 4140 raft_consensus.cc:348] T 9b5aa286c46445a381e581dff1798ed8 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.152796 4142 raft_consensus.cc:374] T 2a08d03748e94ebaab74067281c8f8a2 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.152796 4140 raft_consensus.cc:374] T 9b5aa286c46445a381e581dff1798ed8 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.152834 4142 raft_consensus.cc:729] T 2a08d03748e94ebaab74067281c8f8a2 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0751e317d08443a18b12c780308f9c12, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.152838 4140 raft_consensus.cc:729] T 9b5aa286c46445a381e581dff1798ed8 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1dd721f44bc432c87c22eb45d0345c4, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.152959 4142 consensus_queue.cc:260] T 2a08d03748e94ebaab74067281c8f8a2 P 0751e317d08443a18b12c780308f9c12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.152956 4140 consensus_queue.cc:260] T 9b5aa286c46445a381e581dff1798ed8 P b1dd721f44bc432c87c22eb45d0345c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.153030 4141 raft_consensus.cc:348] T 1839e64af65e4171a1222704f19d1856 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.153126 4141 raft_consensus.cc:374] T 1839e64af65e4171a1222704f19d1856 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.153162 4141 raft_consensus.cc:729] T 1839e64af65e4171a1222704f19d1856 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76cc6f9adf4c4aceac8d8f7d883deb6e, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.153190 4142 ts_tablet_manager.cc:1398] T 2a08d03748e94ebaab74067281c8f8a2 P 0751e317d08443a18b12c780308f9c12: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:12.153292 4140 ts_tablet_manager.cc:1398] T 9b5aa286c46445a381e581dff1798ed8 P b1dd721f44bc432c87c22eb45d0345c4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:12.153347 4142 tablet_bootstrap.cc:492] T c501775259824616959fab8bca85a93b P 0751e317d08443a18b12c780308f9c12: Bootstrap starting.
I20240502 07:58:12.153424 4140 tablet_bootstrap.cc:492] T e2bf12199a744839a7e40531e53d3aea P b1dd721f44bc432c87c22eb45d0345c4: Bootstrap starting.
I20240502 07:58:12.154145 4142 tablet_bootstrap.cc:654] T c501775259824616959fab8bca85a93b P 0751e317d08443a18b12c780308f9c12: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.154356 4140 tablet_bootstrap.cc:654] T e2bf12199a744839a7e40531e53d3aea P b1dd721f44bc432c87c22eb45d0345c4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.154474 4141 consensus_queue.cc:260] T 1839e64af65e4171a1222704f19d1856 P 76cc6f9adf4c4aceac8d8f7d883deb6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.155303 4140 tablet_bootstrap.cc:492] T e2bf12199a744839a7e40531e53d3aea P b1dd721f44bc432c87c22eb45d0345c4: No bootstrap required, opened a new log
I20240502 07:58:12.155377 4140 ts_tablet_manager.cc:1367] T e2bf12199a744839a7e40531e53d3aea P b1dd721f44bc432c87c22eb45d0345c4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:12.155467 4141 ts_tablet_manager.cc:1398] T 1839e64af65e4171a1222704f19d1856 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:58:12.155594 4142 tablet_bootstrap.cc:492] T c501775259824616959fab8bca85a93b P 0751e317d08443a18b12c780308f9c12: No bootstrap required, opened a new log
I20240502 07:58:12.155608 4141 tablet_bootstrap.cc:492] T 86519f378c294fca8232a94970dd5339 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Bootstrap starting.
I20240502 07:58:12.155656 4142 ts_tablet_manager.cc:1367] T c501775259824616959fab8bca85a93b P 0751e317d08443a18b12c780308f9c12: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:12.155767 4140 raft_consensus.cc:348] T e2bf12199a744839a7e40531e53d3aea P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.155874 4140 raft_consensus.cc:374] T e2bf12199a744839a7e40531e53d3aea P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.155900 4140 raft_consensus.cc:729] T e2bf12199a744839a7e40531e53d3aea P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1dd721f44bc432c87c22eb45d0345c4, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.156019 4140 consensus_queue.cc:260] T e2bf12199a744839a7e40531e53d3aea P b1dd721f44bc432c87c22eb45d0345c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.156127 4142 raft_consensus.cc:348] T c501775259824616959fab8bca85a93b P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.156203 4142 raft_consensus.cc:374] T c501775259824616959fab8bca85a93b P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.156230 4142 raft_consensus.cc:729] T c501775259824616959fab8bca85a93b P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0751e317d08443a18b12c780308f9c12, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.156333 4142 consensus_queue.cc:260] T c501775259824616959fab8bca85a93b P 0751e317d08443a18b12c780308f9c12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.156396 4141 tablet_bootstrap.cc:654] T 86519f378c294fca8232a94970dd5339 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.156553 4142 ts_tablet_manager.cc:1398] T c501775259824616959fab8bca85a93b P 0751e317d08443a18b12c780308f9c12: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:12.156697 4142 tablet_bootstrap.cc:492] T 45fa74d198214dc78d63ce787c1490d6 P 0751e317d08443a18b12c780308f9c12: Bootstrap starting.
I20240502 07:58:12.157335 4141 tablet_bootstrap.cc:492] T 86519f378c294fca8232a94970dd5339 P 76cc6f9adf4c4aceac8d8f7d883deb6e: No bootstrap required, opened a new log
I20240502 07:58:12.157407 4141 ts_tablet_manager.cc:1367] T 86519f378c294fca8232a94970dd5339 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:12.157466 4142 tablet_bootstrap.cc:654] T 45fa74d198214dc78d63ce787c1490d6 P 0751e317d08443a18b12c780308f9c12: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.157512 4140 ts_tablet_manager.cc:1398] T e2bf12199a744839a7e40531e53d3aea P b1dd721f44bc432c87c22eb45d0345c4: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 07:58:12.157776 4141 raft_consensus.cc:348] T 86519f378c294fca8232a94970dd5339 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.157881 4141 raft_consensus.cc:374] T 86519f378c294fca8232a94970dd5339 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.157919 4141 raft_consensus.cc:729] T 86519f378c294fca8232a94970dd5339 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76cc6f9adf4c4aceac8d8f7d883deb6e, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.158035 4140 tablet_bootstrap.cc:492] T e7049163a60b47c9899ffc9ad679df95 P b1dd721f44bc432c87c22eb45d0345c4: Bootstrap starting.
I20240502 07:58:12.158025 4141 consensus_queue.cc:260] T 86519f378c294fca8232a94970dd5339 P 76cc6f9adf4c4aceac8d8f7d883deb6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.158270 4141 ts_tablet_manager.cc:1398] T 86519f378c294fca8232a94970dd5339 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:12.158439 4141 tablet_bootstrap.cc:492] T 45fa74d198214dc78d63ce787c1490d6 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Bootstrap starting.
I20240502 07:58:12.158797 4140 tablet_bootstrap.cc:654] T e7049163a60b47c9899ffc9ad679df95 P b1dd721f44bc432c87c22eb45d0345c4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.159385 4141 tablet_bootstrap.cc:654] T 45fa74d198214dc78d63ce787c1490d6 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.159871 4142 tablet_bootstrap.cc:492] T 45fa74d198214dc78d63ce787c1490d6 P 0751e317d08443a18b12c780308f9c12: No bootstrap required, opened a new log
I20240502 07:58:12.159953 4142 ts_tablet_manager.cc:1367] T 45fa74d198214dc78d63ce787c1490d6 P 0751e317d08443a18b12c780308f9c12: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:58:12.160306 4142 raft_consensus.cc:348] T 45fa74d198214dc78d63ce787c1490d6 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.160396 4142 raft_consensus.cc:374] T 45fa74d198214dc78d63ce787c1490d6 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.160415 4142 raft_consensus.cc:729] T 45fa74d198214dc78d63ce787c1490d6 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0751e317d08443a18b12c780308f9c12, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.160516 4142 consensus_queue.cc:260] T 45fa74d198214dc78d63ce787c1490d6 P 0751e317d08443a18b12c780308f9c12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.160578 4141 tablet_bootstrap.cc:492] T 45fa74d198214dc78d63ce787c1490d6 P 76cc6f9adf4c4aceac8d8f7d883deb6e: No bootstrap required, opened a new log
I20240502 07:58:12.160671 4141 ts_tablet_manager.cc:1367] T 45fa74d198214dc78d63ce787c1490d6 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:12.160725 4142 ts_tablet_manager.cc:1398] T 45fa74d198214dc78d63ce787c1490d6 P 0751e317d08443a18b12c780308f9c12: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:12.160917 4142 tablet_bootstrap.cc:492] T 94701214c7954905874450dcfa9a678b P 0751e317d08443a18b12c780308f9c12: Bootstrap starting.
I20240502 07:58:12.161104 4141 raft_consensus.cc:348] T 45fa74d198214dc78d63ce787c1490d6 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.161213 4141 raft_consensus.cc:374] T 45fa74d198214dc78d63ce787c1490d6 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.161254 4141 raft_consensus.cc:729] T 45fa74d198214dc78d63ce787c1490d6 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76cc6f9adf4c4aceac8d8f7d883deb6e, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.161473 4141 consensus_queue.cc:260] T 45fa74d198214dc78d63ce787c1490d6 P 76cc6f9adf4c4aceac8d8f7d883deb6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.161679 4142 tablet_bootstrap.cc:654] T 94701214c7954905874450dcfa9a678b P 0751e317d08443a18b12c780308f9c12: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.161698 4141 ts_tablet_manager.cc:1398] T 45fa74d198214dc78d63ce787c1490d6 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:12.161828 4141 tablet_bootstrap.cc:492] T a3bb326555d747479c66254d5c592b57 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Bootstrap starting.
I20240502 07:58:12.162575 4141 tablet_bootstrap.cc:654] T a3bb326555d747479c66254d5c592b57 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.162637 4142 tablet_bootstrap.cc:492] T 94701214c7954905874450dcfa9a678b P 0751e317d08443a18b12c780308f9c12: No bootstrap required, opened a new log
I20240502 07:58:12.162719 4142 ts_tablet_manager.cc:1367] T 94701214c7954905874450dcfa9a678b P 0751e317d08443a18b12c780308f9c12: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:12.163148 4142 raft_consensus.cc:348] T 94701214c7954905874450dcfa9a678b P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.163245 4142 raft_consensus.cc:374] T 94701214c7954905874450dcfa9a678b P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.163276 4142 raft_consensus.cc:729] T 94701214c7954905874450dcfa9a678b P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0751e317d08443a18b12c780308f9c12, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.163379 4142 consensus_queue.cc:260] T 94701214c7954905874450dcfa9a678b P 0751e317d08443a18b12c780308f9c12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.163630 4141 tablet_bootstrap.cc:492] T a3bb326555d747479c66254d5c592b57 P 76cc6f9adf4c4aceac8d8f7d883deb6e: No bootstrap required, opened a new log
I20240502 07:58:12.163669 4142 ts_tablet_manager.cc:1398] T 94701214c7954905874450dcfa9a678b P 0751e317d08443a18b12c780308f9c12: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:12.163700 4141 ts_tablet_manager.cc:1367] T a3bb326555d747479c66254d5c592b57 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:12.163841 4142 tablet_bootstrap.cc:492] T 1839e64af65e4171a1222704f19d1856 P 0751e317d08443a18b12c780308f9c12: Bootstrap starting.
I20240502 07:58:12.164044 4141 raft_consensus.cc:348] T a3bb326555d747479c66254d5c592b57 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.164134 4141 raft_consensus.cc:374] T a3bb326555d747479c66254d5c592b57 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.164167 4141 raft_consensus.cc:729] T a3bb326555d747479c66254d5c592b57 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76cc6f9adf4c4aceac8d8f7d883deb6e, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.164274 4141 consensus_queue.cc:260] T a3bb326555d747479c66254d5c592b57 P 76cc6f9adf4c4aceac8d8f7d883deb6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.164541 4141 ts_tablet_manager.cc:1398] T a3bb326555d747479c66254d5c592b57 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:12.164618 4142 tablet_bootstrap.cc:654] T 1839e64af65e4171a1222704f19d1856 P 0751e317d08443a18b12c780308f9c12: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.164677 4141 tablet_bootstrap.cc:492] T 9b5aa286c46445a381e581dff1798ed8 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Bootstrap starting.
I20240502 07:58:12.165328 4141 tablet_bootstrap.cc:654] T 9b5aa286c46445a381e581dff1798ed8 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.165647 4142 tablet_bootstrap.cc:492] T 1839e64af65e4171a1222704f19d1856 P 0751e317d08443a18b12c780308f9c12: No bootstrap required, opened a new log
I20240502 07:58:12.165729 4142 ts_tablet_manager.cc:1367] T 1839e64af65e4171a1222704f19d1856 P 0751e317d08443a18b12c780308f9c12: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:12.166230 4141 tablet_bootstrap.cc:492] T 9b5aa286c46445a381e581dff1798ed8 P 76cc6f9adf4c4aceac8d8f7d883deb6e: No bootstrap required, opened a new log
I20240502 07:58:12.166345 4141 ts_tablet_manager.cc:1367] T 9b5aa286c46445a381e581dff1798ed8 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:12.166705 4140 tablet_bootstrap.cc:492] T e7049163a60b47c9899ffc9ad679df95 P b1dd721f44bc432c87c22eb45d0345c4: No bootstrap required, opened a new log
I20240502 07:58:12.166468 4142 raft_consensus.cc:348] T 1839e64af65e4171a1222704f19d1856 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.166774 4142 raft_consensus.cc:374] T 1839e64af65e4171a1222704f19d1856 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.166791 4140 ts_tablet_manager.cc:1367] T e7049163a60b47c9899ffc9ad679df95 P b1dd721f44bc432c87c22eb45d0345c4: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.001s
I20240502 07:58:12.166839 4142 raft_consensus.cc:729] T 1839e64af65e4171a1222704f19d1856 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0751e317d08443a18b12c780308f9c12, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.166924 4141 raft_consensus.cc:348] T 9b5aa286c46445a381e581dff1798ed8 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.166990 4142 consensus_queue.cc:260] T 1839e64af65e4171a1222704f19d1856 P 0751e317d08443a18b12c780308f9c12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.167069 4141 raft_consensus.cc:374] T 9b5aa286c46445a381e581dff1798ed8 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.167188 4141 raft_consensus.cc:729] T 9b5aa286c46445a381e581dff1798ed8 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76cc6f9adf4c4aceac8d8f7d883deb6e, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.167217 4140 raft_consensus.cc:348] T e7049163a60b47c9899ffc9ad679df95 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.167304 4140 raft_consensus.cc:374] T e7049163a60b47c9899ffc9ad679df95 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.167340 4140 raft_consensus.cc:729] T e7049163a60b47c9899ffc9ad679df95 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1dd721f44bc432c87c22eb45d0345c4, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.167447 4141 consensus_queue.cc:260] T 9b5aa286c46445a381e581dff1798ed8 P 76cc6f9adf4c4aceac8d8f7d883deb6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.167641 4142 ts_tablet_manager.cc:1398] T 1839e64af65e4171a1222704f19d1856 P 0751e317d08443a18b12c780308f9c12: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:12.167810 4142 tablet_bootstrap.cc:492] T c7f79b16b7f949f0aa891ef70bdf6608 P 0751e317d08443a18b12c780308f9c12: Bootstrap starting.
I20240502 07:58:12.167819 4141 ts_tablet_manager.cc:1398] T 9b5aa286c46445a381e581dff1798ed8 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:12.167543 4140 consensus_queue.cc:260] T e7049163a60b47c9899ffc9ad679df95 P b1dd721f44bc432c87c22eb45d0345c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.168167 4141 tablet_bootstrap.cc:492] T c7f79b16b7f949f0aa891ef70bdf6608 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Bootstrap starting.
I20240502 07:58:12.168910 4142 tablet_bootstrap.cc:654] T c7f79b16b7f949f0aa891ef70bdf6608 P 0751e317d08443a18b12c780308f9c12: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.169165 4141 tablet_bootstrap.cc:654] T c7f79b16b7f949f0aa891ef70bdf6608 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.170069 4142 tablet_bootstrap.cc:492] T c7f79b16b7f949f0aa891ef70bdf6608 P 0751e317d08443a18b12c780308f9c12: No bootstrap required, opened a new log
I20240502 07:58:12.170178 4142 ts_tablet_manager.cc:1367] T c7f79b16b7f949f0aa891ef70bdf6608 P 0751e317d08443a18b12c780308f9c12: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:12.170214 4140 ts_tablet_manager.cc:1398] T e7049163a60b47c9899ffc9ad679df95 P b1dd721f44bc432c87c22eb45d0345c4: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:58:12.170468 4140 tablet_bootstrap.cc:492] T a3bb326555d747479c66254d5c592b57 P b1dd721f44bc432c87c22eb45d0345c4: Bootstrap starting.
I20240502 07:58:12.170614 4142 raft_consensus.cc:348] T c7f79b16b7f949f0aa891ef70bdf6608 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.170715 4142 raft_consensus.cc:374] T c7f79b16b7f949f0aa891ef70bdf6608 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.170755 4142 raft_consensus.cc:729] T c7f79b16b7f949f0aa891ef70bdf6608 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0751e317d08443a18b12c780308f9c12, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.170859 4142 consensus_queue.cc:260] T c7f79b16b7f949f0aa891ef70bdf6608 P 0751e317d08443a18b12c780308f9c12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.171114 4142 ts_tablet_manager.cc:1398] T c7f79b16b7f949f0aa891ef70bdf6608 P 0751e317d08443a18b12c780308f9c12: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:12.171255 4142 tablet_bootstrap.cc:492] T a3bb326555d747479c66254d5c592b57 P 0751e317d08443a18b12c780308f9c12: Bootstrap starting.
I20240502 07:58:12.171311 4140 tablet_bootstrap.cc:654] T a3bb326555d747479c66254d5c592b57 P b1dd721f44bc432c87c22eb45d0345c4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.172029 4142 tablet_bootstrap.cc:654] T a3bb326555d747479c66254d5c592b57 P 0751e317d08443a18b12c780308f9c12: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.172257 4140 tablet_bootstrap.cc:492] T a3bb326555d747479c66254d5c592b57 P b1dd721f44bc432c87c22eb45d0345c4: No bootstrap required, opened a new log
I20240502 07:58:12.172325 4140 ts_tablet_manager.cc:1367] T a3bb326555d747479c66254d5c592b57 P b1dd721f44bc432c87c22eb45d0345c4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:12.172391 4141 tablet_bootstrap.cc:492] T c7f79b16b7f949f0aa891ef70bdf6608 P 76cc6f9adf4c4aceac8d8f7d883deb6e: No bootstrap required, opened a new log
I20240502 07:58:12.172456 4141 ts_tablet_manager.cc:1367] T c7f79b16b7f949f0aa891ef70bdf6608 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:58:12.172713 4140 raft_consensus.cc:348] T a3bb326555d747479c66254d5c592b57 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.172792 4140 raft_consensus.cc:374] T a3bb326555d747479c66254d5c592b57 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.172821 4140 raft_consensus.cc:729] T a3bb326555d747479c66254d5c592b57 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1dd721f44bc432c87c22eb45d0345c4, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.172785 4141 raft_consensus.cc:348] T c7f79b16b7f949f0aa891ef70bdf6608 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.172860 4141 raft_consensus.cc:374] T c7f79b16b7f949f0aa891ef70bdf6608 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.172899 4141 raft_consensus.cc:729] T c7f79b16b7f949f0aa891ef70bdf6608 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76cc6f9adf4c4aceac8d8f7d883deb6e, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.172951 4142 tablet_bootstrap.cc:492] T a3bb326555d747479c66254d5c592b57 P 0751e317d08443a18b12c780308f9c12: No bootstrap required, opened a new log
I20240502 07:58:12.172945 4140 consensus_queue.cc:260] T a3bb326555d747479c66254d5c592b57 P b1dd721f44bc432c87c22eb45d0345c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.173017 4142 ts_tablet_manager.cc:1367] T a3bb326555d747479c66254d5c592b57 P 0751e317d08443a18b12c780308f9c12: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:12.172994 4141 consensus_queue.cc:260] T c7f79b16b7f949f0aa891ef70bdf6608 P 76cc6f9adf4c4aceac8d8f7d883deb6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.173172 4140 ts_tablet_manager.cc:1398] T a3bb326555d747479c66254d5c592b57 P b1dd721f44bc432c87c22eb45d0345c4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:12.173327 4140 tablet_bootstrap.cc:492] T b99f9c244e014973a024cb91904a64f8 P b1dd721f44bc432c87c22eb45d0345c4: Bootstrap starting.
I20240502 07:58:12.173394 4142 raft_consensus.cc:348] T a3bb326555d747479c66254d5c592b57 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.173478 4142 raft_consensus.cc:374] T a3bb326555d747479c66254d5c592b57 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.173509 4142 raft_consensus.cc:729] T a3bb326555d747479c66254d5c592b57 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0751e317d08443a18b12c780308f9c12, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.173538 4141 ts_tablet_manager.cc:1398] T c7f79b16b7f949f0aa891ef70bdf6608 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:12.173686 4141 tablet_bootstrap.cc:492] T 23d6b429a6374ccd9e5e529186fdcea9 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Bootstrap starting.
I20240502 07:58:12.173681 4142 consensus_queue.cc:260] T a3bb326555d747479c66254d5c592b57 P 0751e317d08443a18b12c780308f9c12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.173930 4142 ts_tablet_manager.cc:1398] T a3bb326555d747479c66254d5c592b57 P 0751e317d08443a18b12c780308f9c12: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:12.174089 4142 tablet_bootstrap.cc:492] T 9b5aa286c46445a381e581dff1798ed8 P 0751e317d08443a18b12c780308f9c12: Bootstrap starting.
I20240502 07:58:12.174536 4141 tablet_bootstrap.cc:654] T 23d6b429a6374ccd9e5e529186fdcea9 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.174882 4140 tablet_bootstrap.cc:654] T b99f9c244e014973a024cb91904a64f8 P b1dd721f44bc432c87c22eb45d0345c4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.174947 4142 tablet_bootstrap.cc:654] T 9b5aa286c46445a381e581dff1798ed8 P 0751e317d08443a18b12c780308f9c12: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.176028 4142 tablet_bootstrap.cc:492] T 9b5aa286c46445a381e581dff1798ed8 P 0751e317d08443a18b12c780308f9c12: No bootstrap required, opened a new log
I20240502 07:58:12.176101 4142 ts_tablet_manager.cc:1367] T 9b5aa286c46445a381e581dff1798ed8 P 0751e317d08443a18b12c780308f9c12: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:12.176491 4142 raft_consensus.cc:348] T 9b5aa286c46445a381e581dff1798ed8 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.176580 4142 raft_consensus.cc:374] T 9b5aa286c46445a381e581dff1798ed8 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.176610 4142 raft_consensus.cc:729] T 9b5aa286c46445a381e581dff1798ed8 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0751e317d08443a18b12c780308f9c12, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.176714 4142 consensus_queue.cc:260] T 9b5aa286c46445a381e581dff1798ed8 P 0751e317d08443a18b12c780308f9c12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.176967 4142 ts_tablet_manager.cc:1398] T 9b5aa286c46445a381e581dff1798ed8 P 0751e317d08443a18b12c780308f9c12: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:12.177122 4142 tablet_bootstrap.cc:492] T 87abd4c4107e4a4aa7dfdb55ee00b26f P 0751e317d08443a18b12c780308f9c12: Bootstrap starting.
I20240502 07:58:12.177426 4141 tablet_bootstrap.cc:492] T 23d6b429a6374ccd9e5e529186fdcea9 P 76cc6f9adf4c4aceac8d8f7d883deb6e: No bootstrap required, opened a new log
I20240502 07:58:12.177510 4141 ts_tablet_manager.cc:1367] T 23d6b429a6374ccd9e5e529186fdcea9 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 07:58:12.177721 4140 tablet_bootstrap.cc:492] T b99f9c244e014973a024cb91904a64f8 P b1dd721f44bc432c87c22eb45d0345c4: No bootstrap required, opened a new log
I20240502 07:58:12.177793 4140 ts_tablet_manager.cc:1367] T b99f9c244e014973a024cb91904a64f8 P b1dd721f44bc432c87c22eb45d0345c4: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:58:12.177904 4141 raft_consensus.cc:348] T 23d6b429a6374ccd9e5e529186fdcea9 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.177986 4141 raft_consensus.cc:374] T 23d6b429a6374ccd9e5e529186fdcea9 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.178012 4142 tablet_bootstrap.cc:654] T 87abd4c4107e4a4aa7dfdb55ee00b26f P 0751e317d08443a18b12c780308f9c12: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.178016 4141 raft_consensus.cc:729] T 23d6b429a6374ccd9e5e529186fdcea9 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76cc6f9adf4c4aceac8d8f7d883deb6e, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.178130 4141 consensus_queue.cc:260] T 23d6b429a6374ccd9e5e529186fdcea9 P 76cc6f9adf4c4aceac8d8f7d883deb6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.178160 4140 raft_consensus.cc:348] T b99f9c244e014973a024cb91904a64f8 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.178246 4140 raft_consensus.cc:374] T b99f9c244e014973a024cb91904a64f8 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.178274 4140 raft_consensus.cc:729] T b99f9c244e014973a024cb91904a64f8 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1dd721f44bc432c87c22eb45d0345c4, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.178359 4141 ts_tablet_manager.cc:1398] T 23d6b429a6374ccd9e5e529186fdcea9 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:12.178385 4140 consensus_queue.cc:260] T b99f9c244e014973a024cb91904a64f8 P b1dd721f44bc432c87c22eb45d0345c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.178597 4141 tablet_bootstrap.cc:492] T 11e2dc546eb3466fab164bfcb0bac310 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Bootstrap starting.
I20240502 07:58:12.178617 4140 ts_tablet_manager.cc:1398] T b99f9c244e014973a024cb91904a64f8 P b1dd721f44bc432c87c22eb45d0345c4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:12.178887 4140 tablet_bootstrap.cc:492] T 11e2dc546eb3466fab164bfcb0bac310 P b1dd721f44bc432c87c22eb45d0345c4: Bootstrap starting.
I20240502 07:58:12.179383 4141 tablet_bootstrap.cc:654] T 11e2dc546eb3466fab164bfcb0bac310 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.179587 4142 tablet_bootstrap.cc:492] T 87abd4c4107e4a4aa7dfdb55ee00b26f P 0751e317d08443a18b12c780308f9c12: No bootstrap required, opened a new log
I20240502 07:58:12.179653 4142 ts_tablet_manager.cc:1367] T 87abd4c4107e4a4aa7dfdb55ee00b26f P 0751e317d08443a18b12c780308f9c12: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:58:12.179742 4140 tablet_bootstrap.cc:654] T 11e2dc546eb3466fab164bfcb0bac310 P b1dd721f44bc432c87c22eb45d0345c4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.180002 4142 raft_consensus.cc:348] T 87abd4c4107e4a4aa7dfdb55ee00b26f P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.180083 4142 raft_consensus.cc:374] T 87abd4c4107e4a4aa7dfdb55ee00b26f P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.180110 4142 raft_consensus.cc:729] T 87abd4c4107e4a4aa7dfdb55ee00b26f P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0751e317d08443a18b12c780308f9c12, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.180222 4142 consensus_queue.cc:260] T 87abd4c4107e4a4aa7dfdb55ee00b26f P 0751e317d08443a18b12c780308f9c12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.180312 4141 tablet_bootstrap.cc:492] T 11e2dc546eb3466fab164bfcb0bac310 P 76cc6f9adf4c4aceac8d8f7d883deb6e: No bootstrap required, opened a new log
I20240502 07:58:12.180389 4141 ts_tablet_manager.cc:1367] T 11e2dc546eb3466fab164bfcb0bac310 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:12.180469 4142 ts_tablet_manager.cc:1398] T 87abd4c4107e4a4aa7dfdb55ee00b26f P 0751e317d08443a18b12c780308f9c12: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:12.180647 4142 tablet_bootstrap.cc:492] T e7049163a60b47c9899ffc9ad679df95 P 0751e317d08443a18b12c780308f9c12: Bootstrap starting.
I20240502 07:58:12.180748 4141 raft_consensus.cc:348] T 11e2dc546eb3466fab164bfcb0bac310 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } }
I20240502 07:58:12.180861 4141 raft_consensus.cc:374] T 11e2dc546eb3466fab164bfcb0bac310 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.180895 4141 raft_consensus.cc:729] T 11e2dc546eb3466fab164bfcb0bac310 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76cc6f9adf4c4aceac8d8f7d883deb6e, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.181015 4141 consensus_queue.cc:260] T 11e2dc546eb3466fab164bfcb0bac310 P 76cc6f9adf4c4aceac8d8f7d883deb6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } }
I20240502 07:58:12.181236 4141 ts_tablet_manager.cc:1398] T 11e2dc546eb3466fab164bfcb0bac310 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:12.181382 4141 tablet_bootstrap.cc:492] T 87abd4c4107e4a4aa7dfdb55ee00b26f P 76cc6f9adf4c4aceac8d8f7d883deb6e: Bootstrap starting.
I20240502 07:58:12.181442 4142 tablet_bootstrap.cc:654] T e7049163a60b47c9899ffc9ad679df95 P 0751e317d08443a18b12c780308f9c12: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.182138 4141 tablet_bootstrap.cc:654] T 87abd4c4107e4a4aa7dfdb55ee00b26f P 76cc6f9adf4c4aceac8d8f7d883deb6e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.182447 4142 tablet_bootstrap.cc:492] T e7049163a60b47c9899ffc9ad679df95 P 0751e317d08443a18b12c780308f9c12: No bootstrap required, opened a new log
I20240502 07:58:12.182520 4142 ts_tablet_manager.cc:1367] T e7049163a60b47c9899ffc9ad679df95 P 0751e317d08443a18b12c780308f9c12: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:12.182886 4142 raft_consensus.cc:348] T e7049163a60b47c9899ffc9ad679df95 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.182976 4142 raft_consensus.cc:374] T e7049163a60b47c9899ffc9ad679df95 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.183007 4142 raft_consensus.cc:729] T e7049163a60b47c9899ffc9ad679df95 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0751e317d08443a18b12c780308f9c12, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.183362 4142 consensus_queue.cc:260] T e7049163a60b47c9899ffc9ad679df95 P 0751e317d08443a18b12c780308f9c12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.183686 4141 tablet_bootstrap.cc:492] T 87abd4c4107e4a4aa7dfdb55ee00b26f P 76cc6f9adf4c4aceac8d8f7d883deb6e: No bootstrap required, opened a new log
I20240502 07:58:12.183679 4142 ts_tablet_manager.cc:1398] T e7049163a60b47c9899ffc9ad679df95 P 0751e317d08443a18b12c780308f9c12: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:12.183755 4141 ts_tablet_manager.cc:1367] T 87abd4c4107e4a4aa7dfdb55ee00b26f P 76cc6f9adf4c4aceac8d8f7d883deb6e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:12.183825 4142 tablet_bootstrap.cc:492] T 11e2dc546eb3466fab164bfcb0bac310 P 0751e317d08443a18b12c780308f9c12: Bootstrap starting.
I20240502 07:58:12.184134 4141 raft_consensus.cc:348] T 87abd4c4107e4a4aa7dfdb55ee00b26f P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.184228 4141 raft_consensus.cc:374] T 87abd4c4107e4a4aa7dfdb55ee00b26f P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.184257 4141 raft_consensus.cc:729] T 87abd4c4107e4a4aa7dfdb55ee00b26f P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76cc6f9adf4c4aceac8d8f7d883deb6e, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.184376 4141 consensus_queue.cc:260] T 87abd4c4107e4a4aa7dfdb55ee00b26f P 76cc6f9adf4c4aceac8d8f7d883deb6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.184593 4142 tablet_bootstrap.cc:654] T 11e2dc546eb3466fab164bfcb0bac310 P 0751e317d08443a18b12c780308f9c12: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.184619 4141 ts_tablet_manager.cc:1398] T 87abd4c4107e4a4aa7dfdb55ee00b26f P 76cc6f9adf4c4aceac8d8f7d883deb6e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:12.184892 4140 tablet_bootstrap.cc:492] T 11e2dc546eb3466fab164bfcb0bac310 P b1dd721f44bc432c87c22eb45d0345c4: No bootstrap required, opened a new log
I20240502 07:58:12.184969 4140 ts_tablet_manager.cc:1367] T 11e2dc546eb3466fab164bfcb0bac310 P b1dd721f44bc432c87c22eb45d0345c4: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240502 07:58:12.185348 4140 raft_consensus.cc:348] T 11e2dc546eb3466fab164bfcb0bac310 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } }
I20240502 07:58:12.185438 4140 raft_consensus.cc:374] T 11e2dc546eb3466fab164bfcb0bac310 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.185459 4142 tablet_bootstrap.cc:492] T 11e2dc546eb3466fab164bfcb0bac310 P 0751e317d08443a18b12c780308f9c12: No bootstrap required, opened a new log
I20240502 07:58:12.185467 4140 raft_consensus.cc:729] T 11e2dc546eb3466fab164bfcb0bac310 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1dd721f44bc432c87c22eb45d0345c4, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.185510 4142 ts_tablet_manager.cc:1367] T 11e2dc546eb3466fab164bfcb0bac310 P 0751e317d08443a18b12c780308f9c12: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:12.185564 4140 consensus_queue.cc:260] T 11e2dc546eb3466fab164bfcb0bac310 P b1dd721f44bc432c87c22eb45d0345c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } }
I20240502 07:58:12.185794 4140 ts_tablet_manager.cc:1398] T 11e2dc546eb3466fab164bfcb0bac310 P b1dd721f44bc432c87c22eb45d0345c4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:12.185814 4142 raft_consensus.cc:348] T 11e2dc546eb3466fab164bfcb0bac310 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } }
I20240502 07:58:12.185902 4142 raft_consensus.cc:374] T 11e2dc546eb3466fab164bfcb0bac310 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.186029 4142 raft_consensus.cc:729] T 11e2dc546eb3466fab164bfcb0bac310 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0751e317d08443a18b12c780308f9c12, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.186097 4140 tablet_bootstrap.cc:492] T 45fa74d198214dc78d63ce787c1490d6 P b1dd721f44bc432c87c22eb45d0345c4: Bootstrap starting.
I20240502 07:58:12.186161 4142 consensus_queue.cc:260] T 11e2dc546eb3466fab164bfcb0bac310 P 0751e317d08443a18b12c780308f9c12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } }
I20240502 07:58:12.186406 4142 ts_tablet_manager.cc:1398] T 11e2dc546eb3466fab164bfcb0bac310 P 0751e317d08443a18b12c780308f9c12: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:12.186522 4142 tablet_bootstrap.cc:492] T 02538cc46cb6465d927569ed749c2154 P 0751e317d08443a18b12c780308f9c12: Bootstrap starting.
I20240502 07:58:12.186846 4140 tablet_bootstrap.cc:654] T 45fa74d198214dc78d63ce787c1490d6 P b1dd721f44bc432c87c22eb45d0345c4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.187242 4142 tablet_bootstrap.cc:654] T 02538cc46cb6465d927569ed749c2154 P 0751e317d08443a18b12c780308f9c12: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.187842 4140 tablet_bootstrap.cc:492] T 45fa74d198214dc78d63ce787c1490d6 P b1dd721f44bc432c87c22eb45d0345c4: No bootstrap required, opened a new log
I20240502 07:58:12.187928 4140 ts_tablet_manager.cc:1367] T 45fa74d198214dc78d63ce787c1490d6 P b1dd721f44bc432c87c22eb45d0345c4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:12.188298 4142 tablet_bootstrap.cc:492] T 02538cc46cb6465d927569ed749c2154 P 0751e317d08443a18b12c780308f9c12: No bootstrap required, opened a new log
I20240502 07:58:12.188293 4140 raft_consensus.cc:348] T 45fa74d198214dc78d63ce787c1490d6 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.188369 4142 ts_tablet_manager.cc:1367] T 02538cc46cb6465d927569ed749c2154 P 0751e317d08443a18b12c780308f9c12: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:12.188378 4140 raft_consensus.cc:374] T 45fa74d198214dc78d63ce787c1490d6 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.188406 4140 raft_consensus.cc:729] T 45fa74d198214dc78d63ce787c1490d6 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1dd721f44bc432c87c22eb45d0345c4, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.188509 4140 consensus_queue.cc:260] T 45fa74d198214dc78d63ce787c1490d6 P b1dd721f44bc432c87c22eb45d0345c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.188694 4142 raft_consensus.cc:348] T 02538cc46cb6465d927569ed749c2154 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.188727 4140 ts_tablet_manager.cc:1398] T 45fa74d198214dc78d63ce787c1490d6 P b1dd721f44bc432c87c22eb45d0345c4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:12.188768 4142 raft_consensus.cc:374] T 02538cc46cb6465d927569ed749c2154 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.188787 4142 raft_consensus.cc:729] T 02538cc46cb6465d927569ed749c2154 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0751e317d08443a18b12c780308f9c12, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.188861 4140 tablet_bootstrap.cc:492] T 02538cc46cb6465d927569ed749c2154 P b1dd721f44bc432c87c22eb45d0345c4: Bootstrap starting.
I20240502 07:58:12.188874 4142 consensus_queue.cc:260] T 02538cc46cb6465d927569ed749c2154 P 0751e317d08443a18b12c780308f9c12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.189103 4142 ts_tablet_manager.cc:1398] T 02538cc46cb6465d927569ed749c2154 P 0751e317d08443a18b12c780308f9c12: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:12.189656 4140 tablet_bootstrap.cc:654] T 02538cc46cb6465d927569ed749c2154 P b1dd721f44bc432c87c22eb45d0345c4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.190840 4140 tablet_bootstrap.cc:492] T 02538cc46cb6465d927569ed749c2154 P b1dd721f44bc432c87c22eb45d0345c4: No bootstrap required, opened a new log
I20240502 07:58:12.190929 4140 ts_tablet_manager.cc:1367] T 02538cc46cb6465d927569ed749c2154 P b1dd721f44bc432c87c22eb45d0345c4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:12.191323 4140 raft_consensus.cc:348] T 02538cc46cb6465d927569ed749c2154 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.191399 4140 raft_consensus.cc:374] T 02538cc46cb6465d927569ed749c2154 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.191418 4140 raft_consensus.cc:729] T 02538cc46cb6465d927569ed749c2154 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1dd721f44bc432c87c22eb45d0345c4, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.191502 4140 consensus_queue.cc:260] T 02538cc46cb6465d927569ed749c2154 P b1dd721f44bc432c87c22eb45d0345c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.191726 4140 ts_tablet_manager.cc:1398] T 02538cc46cb6465d927569ed749c2154 P b1dd721f44bc432c87c22eb45d0345c4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:12.191849 4140 tablet_bootstrap.cc:492] T c501775259824616959fab8bca85a93b P b1dd721f44bc432c87c22eb45d0345c4: Bootstrap starting.
I20240502 07:58:12.192160 4151 raft_consensus.cc:482] T b99f9c244e014973a024cb91904a64f8 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:12.192318 4151 raft_consensus.cc:504] T b99f9c244e014973a024cb91904a64f8 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.192562 4140 tablet_bootstrap.cc:654] T c501775259824616959fab8bca85a93b P b1dd721f44bc432c87c22eb45d0345c4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.193627 4151 leader_election.cc:288] T b99f9c244e014973a024cb91904a64f8 P 0751e317d08443a18b12c780308f9c12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1dd721f44bc432c87c22eb45d0345c4 (127.3.139.66:46179), 76cc6f9adf4c4aceac8d8f7d883deb6e (127.3.139.65:35675)
I20240502 07:58:12.193719 4140 tablet_bootstrap.cc:492] T c501775259824616959fab8bca85a93b P b1dd721f44bc432c87c22eb45d0345c4: No bootstrap required, opened a new log
I20240502 07:58:12.193783 4140 ts_tablet_manager.cc:1367] T c501775259824616959fab8bca85a93b P b1dd721f44bc432c87c22eb45d0345c4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:12.194144 4140 raft_consensus.cc:348] T c501775259824616959fab8bca85a93b P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.194344 4140 raft_consensus.cc:374] T c501775259824616959fab8bca85a93b P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.194383 4140 raft_consensus.cc:729] T c501775259824616959fab8bca85a93b P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1dd721f44bc432c87c22eb45d0345c4, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.194474 4140 consensus_queue.cc:260] T c501775259824616959fab8bca85a93b P b1dd721f44bc432c87c22eb45d0345c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.194703 4140 ts_tablet_manager.cc:1398] T c501775259824616959fab8bca85a93b P b1dd721f44bc432c87c22eb45d0345c4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:12.195358 4140 tablet_bootstrap.cc:492] T 23d6b429a6374ccd9e5e529186fdcea9 P b1dd721f44bc432c87c22eb45d0345c4: Bootstrap starting.
I20240502 07:58:12.196233 4140 tablet_bootstrap.cc:654] T 23d6b429a6374ccd9e5e529186fdcea9 P b1dd721f44bc432c87c22eb45d0345c4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.198186 4140 tablet_bootstrap.cc:492] T 23d6b429a6374ccd9e5e529186fdcea9 P b1dd721f44bc432c87c22eb45d0345c4: No bootstrap required, opened a new log
I20240502 07:58:12.198267 4140 ts_tablet_manager.cc:1367] T 23d6b429a6374ccd9e5e529186fdcea9 P b1dd721f44bc432c87c22eb45d0345c4: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:58:12.198545 3945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b99f9c244e014973a024cb91904a64f8" candidate_uuid: "0751e317d08443a18b12c780308f9c12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1dd721f44bc432c87c22eb45d0345c4" is_pre_election: true
I20240502 07:58:12.198603 3814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b99f9c244e014973a024cb91904a64f8" candidate_uuid: "0751e317d08443a18b12c780308f9c12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" is_pre_election: true
I20240502 07:58:12.198622 4140 raft_consensus.cc:348] T 23d6b429a6374ccd9e5e529186fdcea9 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.198694 4140 raft_consensus.cc:374] T 23d6b429a6374ccd9e5e529186fdcea9 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.198720 4140 raft_consensus.cc:729] T 23d6b429a6374ccd9e5e529186fdcea9 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1dd721f44bc432c87c22eb45d0345c4, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.198773 3945 raft_consensus.cc:2454] T b99f9c244e014973a024cb91904a64f8 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0751e317d08443a18b12c780308f9c12 in term 0.
I20240502 07:58:12.198843 3814 raft_consensus.cc:2454] T b99f9c244e014973a024cb91904a64f8 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0751e317d08443a18b12c780308f9c12 in term 0.
I20240502 07:58:12.198827 4140 consensus_queue.cc:260] T 23d6b429a6374ccd9e5e529186fdcea9 P b1dd721f44bc432c87c22eb45d0345c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.199085 4140 ts_tablet_manager.cc:1398] T 23d6b429a6374ccd9e5e529186fdcea9 P b1dd721f44bc432c87c22eb45d0345c4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:12.199239 4140 tablet_bootstrap.cc:492] T 94701214c7954905874450dcfa9a678b P b1dd721f44bc432c87c22eb45d0345c4: Bootstrap starting.
I20240502 07:58:12.199431 4011 leader_election.cc:302] T b99f9c244e014973a024cb91904a64f8 P 0751e317d08443a18b12c780308f9c12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0751e317d08443a18b12c780308f9c12, 76cc6f9adf4c4aceac8d8f7d883deb6e; no voters:
I20240502 07:58:12.199630 4151 raft_consensus.cc:2781] T b99f9c244e014973a024cb91904a64f8 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:12.199700 4151 raft_consensus.cc:482] T b99f9c244e014973a024cb91904a64f8 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:12.199740 4151 raft_consensus.cc:3037] T b99f9c244e014973a024cb91904a64f8 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.199980 4140 tablet_bootstrap.cc:654] T 94701214c7954905874450dcfa9a678b P b1dd721f44bc432c87c22eb45d0345c4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.200723 4151 raft_consensus.cc:504] T b99f9c244e014973a024cb91904a64f8 P 0751e317d08443a18b12c780308f9c12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.201061 4151 leader_election.cc:288] T b99f9c244e014973a024cb91904a64f8 P 0751e317d08443a18b12c780308f9c12 [CANDIDATE]: Term 1 election: Requested vote from peers b1dd721f44bc432c87c22eb45d0345c4 (127.3.139.66:46179), 76cc6f9adf4c4aceac8d8f7d883deb6e (127.3.139.65:35675)
I20240502 07:58:12.201133 4140 tablet_bootstrap.cc:492] T 94701214c7954905874450dcfa9a678b P b1dd721f44bc432c87c22eb45d0345c4: No bootstrap required, opened a new log
I20240502 07:58:12.201197 4140 ts_tablet_manager.cc:1367] T 94701214c7954905874450dcfa9a678b P b1dd721f44bc432c87c22eb45d0345c4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:12.201421 3945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b99f9c244e014973a024cb91904a64f8" candidate_uuid: "0751e317d08443a18b12c780308f9c12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1dd721f44bc432c87c22eb45d0345c4"
I20240502 07:58:12.201457 3814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b99f9c244e014973a024cb91904a64f8" candidate_uuid: "0751e317d08443a18b12c780308f9c12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e"
I20240502 07:58:12.201536 3945 raft_consensus.cc:3037] T b99f9c244e014973a024cb91904a64f8 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.201550 3814 raft_consensus.cc:3037] T b99f9c244e014973a024cb91904a64f8 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.201689 4140 raft_consensus.cc:348] T 94701214c7954905874450dcfa9a678b P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.201779 4140 raft_consensus.cc:374] T 94701214c7954905874450dcfa9a678b P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.201808 4140 raft_consensus.cc:729] T 94701214c7954905874450dcfa9a678b P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1dd721f44bc432c87c22eb45d0345c4, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.201938 4140 consensus_queue.cc:260] T 94701214c7954905874450dcfa9a678b P b1dd721f44bc432c87c22eb45d0345c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.202159 4140 ts_tablet_manager.cc:1398] T 94701214c7954905874450dcfa9a678b P b1dd721f44bc432c87c22eb45d0345c4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:12.202316 4140 tablet_bootstrap.cc:492] T 1839e64af65e4171a1222704f19d1856 P b1dd721f44bc432c87c22eb45d0345c4: Bootstrap starting.
I20240502 07:58:12.202469 3945 raft_consensus.cc:2454] T b99f9c244e014973a024cb91904a64f8 P b1dd721f44bc432c87c22eb45d0345c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0751e317d08443a18b12c780308f9c12 in term 1.
I20240502 07:58:12.202601 3814 raft_consensus.cc:2454] T b99f9c244e014973a024cb91904a64f8 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0751e317d08443a18b12c780308f9c12 in term 1.
I20240502 07:58:12.202795 4011 leader_election.cc:302] T b99f9c244e014973a024cb91904a64f8 P 0751e317d08443a18b12c780308f9c12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0751e317d08443a18b12c780308f9c12, b1dd721f44bc432c87c22eb45d0345c4; no voters:
I20240502 07:58:12.202960 4151 raft_consensus.cc:2781] T b99f9c244e014973a024cb91904a64f8 P 0751e317d08443a18b12c780308f9c12 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:12.203177 4151 raft_consensus.cc:686] T b99f9c244e014973a024cb91904a64f8 P 0751e317d08443a18b12c780308f9c12 [term 1 LEADER]: Becoming Leader. State: Replica: 0751e317d08443a18b12c780308f9c12, State: Running, Role: LEADER
I20240502 07:58:12.203280 4140 tablet_bootstrap.cc:654] T 1839e64af65e4171a1222704f19d1856 P b1dd721f44bc432c87c22eb45d0345c4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.203493 4151 consensus_queue.cc:237] T b99f9c244e014973a024cb91904a64f8 P 0751e317d08443a18b12c780308f9c12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.205415 4151 raft_consensus.cc:482] T c501775259824616959fab8bca85a93b P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:12.205487 4151 raft_consensus.cc:504] T c501775259824616959fab8bca85a93b P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.205735 4140 tablet_bootstrap.cc:492] T 1839e64af65e4171a1222704f19d1856 P b1dd721f44bc432c87c22eb45d0345c4: No bootstrap required, opened a new log
I20240502 07:58:12.205790 4151 leader_election.cc:288] T c501775259824616959fab8bca85a93b P 0751e317d08443a18b12c780308f9c12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76cc6f9adf4c4aceac8d8f7d883deb6e (127.3.139.65:35675), b1dd721f44bc432c87c22eb45d0345c4 (127.3.139.66:46179)
I20240502 07:58:12.205811 4140 ts_tablet_manager.cc:1367] T 1839e64af65e4171a1222704f19d1856 P b1dd721f44bc432c87c22eb45d0345c4: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:58:12.205862 3670 catalog_manager.cc:5484] T b99f9c244e014973a024cb91904a64f8 P 0751e317d08443a18b12c780308f9c12 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0751e317d08443a18b12c780308f9c12 (127.3.139.67). New cstate: current_term: 1 leader_uuid: "0751e317d08443a18b12c780308f9c12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } health_report { overall_health: HEALTHY } } }
I20240502 07:58:12.206128 3814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c501775259824616959fab8bca85a93b" candidate_uuid: "0751e317d08443a18b12c780308f9c12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" is_pre_election: true
I20240502 07:58:12.206128 3945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c501775259824616959fab8bca85a93b" candidate_uuid: "0751e317d08443a18b12c780308f9c12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1dd721f44bc432c87c22eb45d0345c4" is_pre_election: true
I20240502 07:58:12.206233 3945 raft_consensus.cc:2454] T c501775259824616959fab8bca85a93b P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0751e317d08443a18b12c780308f9c12 in term 0.
I20240502 07:58:12.206233 3814 raft_consensus.cc:2454] T c501775259824616959fab8bca85a93b P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0751e317d08443a18b12c780308f9c12 in term 0.
I20240502 07:58:12.206496 4140 raft_consensus.cc:348] T 1839e64af65e4171a1222704f19d1856 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.206564 4140 raft_consensus.cc:374] T 1839e64af65e4171a1222704f19d1856 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.206583 4140 raft_consensus.cc:729] T 1839e64af65e4171a1222704f19d1856 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1dd721f44bc432c87c22eb45d0345c4, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.206566 4011 leader_election.cc:302] T c501775259824616959fab8bca85a93b P 0751e317d08443a18b12c780308f9c12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0751e317d08443a18b12c780308f9c12, b1dd721f44bc432c87c22eb45d0345c4; no voters:
I20240502 07:58:12.206665 4140 consensus_queue.cc:260] T 1839e64af65e4171a1222704f19d1856 P b1dd721f44bc432c87c22eb45d0345c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.206719 4151 raft_consensus.cc:2781] T c501775259824616959fab8bca85a93b P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:12.206763 4151 raft_consensus.cc:482] T c501775259824616959fab8bca85a93b P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:12.206795 4151 raft_consensus.cc:3037] T c501775259824616959fab8bca85a93b P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.206964 4140 ts_tablet_manager.cc:1398] T 1839e64af65e4171a1222704f19d1856 P b1dd721f44bc432c87c22eb45d0345c4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:12.207557 4151 raft_consensus.cc:504] T c501775259824616959fab8bca85a93b P 0751e317d08443a18b12c780308f9c12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.207830 4151 leader_election.cc:288] T c501775259824616959fab8bca85a93b P 0751e317d08443a18b12c780308f9c12 [CANDIDATE]: Term 1 election: Requested vote from peers 76cc6f9adf4c4aceac8d8f7d883deb6e (127.3.139.65:35675), b1dd721f44bc432c87c22eb45d0345c4 (127.3.139.66:46179)
I20240502 07:58:12.208108 3814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c501775259824616959fab8bca85a93b" candidate_uuid: "0751e317d08443a18b12c780308f9c12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e"
I20240502 07:58:12.208207 3814 raft_consensus.cc:3037] T c501775259824616959fab8bca85a93b P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.208483 3945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c501775259824616959fab8bca85a93b" candidate_uuid: "0751e317d08443a18b12c780308f9c12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1dd721f44bc432c87c22eb45d0345c4"
I20240502 07:58:12.208563 3945 raft_consensus.cc:3037] T c501775259824616959fab8bca85a93b P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.208983 3814 raft_consensus.cc:2454] T c501775259824616959fab8bca85a93b P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0751e317d08443a18b12c780308f9c12 in term 1.
I20240502 07:58:12.209265 4011 leader_election.cc:302] T c501775259824616959fab8bca85a93b P 0751e317d08443a18b12c780308f9c12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0751e317d08443a18b12c780308f9c12, 76cc6f9adf4c4aceac8d8f7d883deb6e; no voters:
I20240502 07:58:12.209337 3945 raft_consensus.cc:2454] T c501775259824616959fab8bca85a93b P b1dd721f44bc432c87c22eb45d0345c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0751e317d08443a18b12c780308f9c12 in term 1.
I20240502 07:58:12.209633 4151 raft_consensus.cc:2781] T c501775259824616959fab8bca85a93b P 0751e317d08443a18b12c780308f9c12 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:12.209725 4151 raft_consensus.cc:686] T c501775259824616959fab8bca85a93b P 0751e317d08443a18b12c780308f9c12 [term 1 LEADER]: Becoming Leader. State: Replica: 0751e317d08443a18b12c780308f9c12, State: Running, Role: LEADER
I20240502 07:58:12.209827 4151 consensus_queue.cc:237] T c501775259824616959fab8bca85a93b P 0751e317d08443a18b12c780308f9c12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.211035 3670 catalog_manager.cc:5484] T c501775259824616959fab8bca85a93b P 0751e317d08443a18b12c780308f9c12 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0751e317d08443a18b12c780308f9c12 (127.3.139.67). New cstate: current_term: 1 leader_uuid: "0751e317d08443a18b12c780308f9c12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } health_report { overall_health: HEALTHY } } }
I20240502 07:58:12.212849 4150 raft_consensus.cc:482] T 1839e64af65e4171a1222704f19d1856 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:12.212929 4150 raft_consensus.cc:504] T 1839e64af65e4171a1222704f19d1856 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.213711 4150 leader_election.cc:288] T 1839e64af65e4171a1222704f19d1856 P b1dd721f44bc432c87c22eb45d0345c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76cc6f9adf4c4aceac8d8f7d883deb6e (127.3.139.65:35675), 0751e317d08443a18b12c780308f9c12 (127.3.139.67:33895)
I20240502 07:58:12.214928 4150 raft_consensus.cc:482] T c7f79b16b7f949f0aa891ef70bdf6608 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:12.215008 4150 raft_consensus.cc:504] T c7f79b16b7f949f0aa891ef70bdf6608 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.215343 4150 leader_election.cc:288] T c7f79b16b7f949f0aa891ef70bdf6608 P b1dd721f44bc432c87c22eb45d0345c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76cc6f9adf4c4aceac8d8f7d883deb6e (127.3.139.65:35675), 0751e317d08443a18b12c780308f9c12 (127.3.139.67:33895)
I20240502 07:58:12.217232 3814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1839e64af65e4171a1222704f19d1856" candidate_uuid: "b1dd721f44bc432c87c22eb45d0345c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" is_pre_election: true
I20240502 07:58:12.217290 3813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f79b16b7f949f0aa891ef70bdf6608" candidate_uuid: "b1dd721f44bc432c87c22eb45d0345c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" is_pre_election: true
I20240502 07:58:12.217345 3814 raft_consensus.cc:2454] T 1839e64af65e4171a1222704f19d1856 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1dd721f44bc432c87c22eb45d0345c4 in term 0.
I20240502 07:58:12.217403 3813 raft_consensus.cc:2454] T c7f79b16b7f949f0aa891ef70bdf6608 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1dd721f44bc432c87c22eb45d0345c4 in term 0.
I20240502 07:58:12.217712 3880 leader_election.cc:302] T 1839e64af65e4171a1222704f19d1856 P b1dd721f44bc432c87c22eb45d0345c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76cc6f9adf4c4aceac8d8f7d883deb6e, b1dd721f44bc432c87c22eb45d0345c4; no voters:
I20240502 07:58:12.217715 4076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1839e64af65e4171a1222704f19d1856" candidate_uuid: "b1dd721f44bc432c87c22eb45d0345c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0751e317d08443a18b12c780308f9c12" is_pre_election: true
I20240502 07:58:12.217720 4075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f79b16b7f949f0aa891ef70bdf6608" candidate_uuid: "b1dd721f44bc432c87c22eb45d0345c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0751e317d08443a18b12c780308f9c12" is_pre_election: true
I20240502 07:58:12.217872 4076 raft_consensus.cc:2454] T 1839e64af65e4171a1222704f19d1856 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1dd721f44bc432c87c22eb45d0345c4 in term 0.
I20240502 07:58:12.217871 4075 raft_consensus.cc:2454] T c7f79b16b7f949f0aa891ef70bdf6608 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1dd721f44bc432c87c22eb45d0345c4 in term 0.
I20240502 07:58:12.217924 4150 raft_consensus.cc:2781] T 1839e64af65e4171a1222704f19d1856 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:12.217950 3880 leader_election.cc:302] T c7f79b16b7f949f0aa891ef70bdf6608 P b1dd721f44bc432c87c22eb45d0345c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76cc6f9adf4c4aceac8d8f7d883deb6e, b1dd721f44bc432c87c22eb45d0345c4; no voters:
I20240502 07:58:12.217967 4150 raft_consensus.cc:482] T 1839e64af65e4171a1222704f19d1856 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:12.218024 4150 raft_consensus.cc:3037] T 1839e64af65e4171a1222704f19d1856 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.218194 4202 raft_consensus.cc:2781] T c7f79b16b7f949f0aa891ef70bdf6608 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:12.218245 4202 raft_consensus.cc:482] T c7f79b16b7f949f0aa891ef70bdf6608 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:12.218282 4202 raft_consensus.cc:3037] T c7f79b16b7f949f0aa891ef70bdf6608 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.218991 4150 raft_consensus.cc:504] T 1839e64af65e4171a1222704f19d1856 P b1dd721f44bc432c87c22eb45d0345c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.219075 4202 raft_consensus.cc:504] T c7f79b16b7f949f0aa891ef70bdf6608 P b1dd721f44bc432c87c22eb45d0345c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.219292 4150 leader_election.cc:288] T 1839e64af65e4171a1222704f19d1856 P b1dd721f44bc432c87c22eb45d0345c4 [CANDIDATE]: Term 1 election: Requested vote from peers 76cc6f9adf4c4aceac8d8f7d883deb6e (127.3.139.65:35675), 0751e317d08443a18b12c780308f9c12 (127.3.139.67:33895)
I20240502 07:58:12.219358 4202 leader_election.cc:288] T c7f79b16b7f949f0aa891ef70bdf6608 P b1dd721f44bc432c87c22eb45d0345c4 [CANDIDATE]: Term 1 election: Requested vote from peers 76cc6f9adf4c4aceac8d8f7d883deb6e (127.3.139.65:35675), 0751e317d08443a18b12c780308f9c12 (127.3.139.67:33895)
I20240502 07:58:12.219377 4151 raft_consensus.cc:482] T e7049163a60b47c9899ffc9ad679df95 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:12.219427 4151 raft_consensus.cc:504] T e7049163a60b47c9899ffc9ad679df95 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.219671 4151 leader_election.cc:288] T e7049163a60b47c9899ffc9ad679df95 P 0751e317d08443a18b12c780308f9c12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1dd721f44bc432c87c22eb45d0345c4 (127.3.139.66:46179), 76cc6f9adf4c4aceac8d8f7d883deb6e (127.3.139.65:35675)
I20240502 07:58:12.219722 3813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1839e64af65e4171a1222704f19d1856" candidate_uuid: "b1dd721f44bc432c87c22eb45d0345c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e"
I20240502 07:58:12.219823 3813 raft_consensus.cc:3037] T 1839e64af65e4171a1222704f19d1856 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.219882 4075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1839e64af65e4171a1222704f19d1856" candidate_uuid: "b1dd721f44bc432c87c22eb45d0345c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0751e317d08443a18b12c780308f9c12"
I20240502 07:58:12.219949 3945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7049163a60b47c9899ffc9ad679df95" candidate_uuid: "0751e317d08443a18b12c780308f9c12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1dd721f44bc432c87c22eb45d0345c4" is_pre_election: true
I20240502 07:58:12.219975 4075 raft_consensus.cc:3037] T 1839e64af65e4171a1222704f19d1856 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.220048 3945 raft_consensus.cc:2454] T e7049163a60b47c9899ffc9ad679df95 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0751e317d08443a18b12c780308f9c12 in term 0.
I20240502 07:58:12.220237 3814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f79b16b7f949f0aa891ef70bdf6608" candidate_uuid: "b1dd721f44bc432c87c22eb45d0345c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e"
I20240502 07:58:12.220321 3814 raft_consensus.cc:3037] T c7f79b16b7f949f0aa891ef70bdf6608 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.220311 4011 leader_election.cc:302] T e7049163a60b47c9899ffc9ad679df95 P 0751e317d08443a18b12c780308f9c12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0751e317d08443a18b12c780308f9c12, b1dd721f44bc432c87c22eb45d0345c4; no voters:
I20240502 07:58:12.220461 4151 raft_consensus.cc:2781] T e7049163a60b47c9899ffc9ad679df95 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:12.220499 4151 raft_consensus.cc:482] T e7049163a60b47c9899ffc9ad679df95 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:12.220530 4151 raft_consensus.cc:3037] T e7049163a60b47c9899ffc9ad679df95 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.220666 3813 raft_consensus.cc:2454] T 1839e64af65e4171a1222704f19d1856 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1dd721f44bc432c87c22eb45d0345c4 in term 1.
I20240502 07:58:12.220849 4075 raft_consensus.cc:2454] T 1839e64af65e4171a1222704f19d1856 P 0751e317d08443a18b12c780308f9c12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1dd721f44bc432c87c22eb45d0345c4 in term 1.
I20240502 07:58:12.220893 4076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f79b16b7f949f0aa891ef70bdf6608" candidate_uuid: "b1dd721f44bc432c87c22eb45d0345c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0751e317d08443a18b12c780308f9c12"
I20240502 07:58:12.220979 4076 raft_consensus.cc:3037] T c7f79b16b7f949f0aa891ef70bdf6608 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.221031 3814 raft_consensus.cc:2454] T c7f79b16b7f949f0aa891ef70bdf6608 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1dd721f44bc432c87c22eb45d0345c4 in term 1.
W20240502 07:58:12.221166 3861 tablet.cc:2369] T 02538cc46cb6465d927569ed749c2154 P 76cc6f9adf4c4aceac8d8f7d883deb6e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 07:58:12.221184 4151 raft_consensus.cc:504] T e7049163a60b47c9899ffc9ad679df95 P 0751e317d08443a18b12c780308f9c12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.221441 4151 leader_election.cc:288] T e7049163a60b47c9899ffc9ad679df95 P 0751e317d08443a18b12c780308f9c12 [CANDIDATE]: Term 1 election: Requested vote from peers b1dd721f44bc432c87c22eb45d0345c4 (127.3.139.66:46179), 76cc6f9adf4c4aceac8d8f7d883deb6e (127.3.139.65:35675)
I20240502 07:58:12.221511 3880 leader_election.cc:302] T 1839e64af65e4171a1222704f19d1856 P b1dd721f44bc432c87c22eb45d0345c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76cc6f9adf4c4aceac8d8f7d883deb6e, b1dd721f44bc432c87c22eb45d0345c4; no voters:
I20240502 07:58:12.221657 3880 leader_election.cc:302] T c7f79b16b7f949f0aa891ef70bdf6608 P b1dd721f44bc432c87c22eb45d0345c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76cc6f9adf4c4aceac8d8f7d883deb6e, b1dd721f44bc432c87c22eb45d0345c4; no voters:
I20240502 07:58:12.221799 4150 raft_consensus.cc:2781] T 1839e64af65e4171a1222704f19d1856 P b1dd721f44bc432c87c22eb45d0345c4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:12.221793 4076 raft_consensus.cc:2454] T c7f79b16b7f949f0aa891ef70bdf6608 P 0751e317d08443a18b12c780308f9c12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1dd721f44bc432c87c22eb45d0345c4 in term 1.
I20240502 07:58:12.221975 4150 raft_consensus.cc:686] T 1839e64af65e4171a1222704f19d1856 P b1dd721f44bc432c87c22eb45d0345c4 [term 1 LEADER]: Becoming Leader. State: Replica: b1dd721f44bc432c87c22eb45d0345c4, State: Running, Role: LEADER
I20240502 07:58:12.222075 3945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7049163a60b47c9899ffc9ad679df95" candidate_uuid: "0751e317d08443a18b12c780308f9c12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1dd721f44bc432c87c22eb45d0345c4"
I20240502 07:58:12.222160 3814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7049163a60b47c9899ffc9ad679df95" candidate_uuid: "0751e317d08443a18b12c780308f9c12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" is_pre_election: true
I20240502 07:58:12.222191 3945 raft_consensus.cc:3037] T e7049163a60b47c9899ffc9ad679df95 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.222232 3813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7049163a60b47c9899ffc9ad679df95" candidate_uuid: "0751e317d08443a18b12c780308f9c12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e"
I20240502 07:58:12.222249 3814 raft_consensus.cc:2454] T e7049163a60b47c9899ffc9ad679df95 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0751e317d08443a18b12c780308f9c12 in term 0.
I20240502 07:58:12.222240 4150 consensus_queue.cc:237] T 1839e64af65e4171a1222704f19d1856 P b1dd721f44bc432c87c22eb45d0345c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.222818 3945 raft_consensus.cc:2454] T e7049163a60b47c9899ffc9ad679df95 P b1dd721f44bc432c87c22eb45d0345c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0751e317d08443a18b12c780308f9c12 in term 1.
I20240502 07:58:12.223071 4202 raft_consensus.cc:2781] T c7f79b16b7f949f0aa891ef70bdf6608 P b1dd721f44bc432c87c22eb45d0345c4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:12.223210 4011 leader_election.cc:302] T e7049163a60b47c9899ffc9ad679df95 P 0751e317d08443a18b12c780308f9c12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0751e317d08443a18b12c780308f9c12, b1dd721f44bc432c87c22eb45d0345c4; no voters:
I20240502 07:58:12.223351 4202 raft_consensus.cc:686] T c7f79b16b7f949f0aa891ef70bdf6608 P b1dd721f44bc432c87c22eb45d0345c4 [term 1 LEADER]: Becoming Leader. State: Replica: b1dd721f44bc432c87c22eb45d0345c4, State: Running, Role: LEADER
I20240502 07:58:12.223379 4151 raft_consensus.cc:2781] T e7049163a60b47c9899ffc9ad679df95 P 0751e317d08443a18b12c780308f9c12 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:12.223474 4151 raft_consensus.cc:686] T e7049163a60b47c9899ffc9ad679df95 P 0751e317d08443a18b12c780308f9c12 [term 1 LEADER]: Becoming Leader. State: Replica: 0751e317d08443a18b12c780308f9c12, State: Running, Role: LEADER
I20240502 07:58:12.223583 4151 consensus_queue.cc:237] T e7049163a60b47c9899ffc9ad679df95 P 0751e317d08443a18b12c780308f9c12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.223582 4202 consensus_queue.cc:237] T c7f79b16b7f949f0aa891ef70bdf6608 P b1dd721f44bc432c87c22eb45d0345c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.224589 3670 catalog_manager.cc:5484] T 1839e64af65e4171a1222704f19d1856 P b1dd721f44bc432c87c22eb45d0345c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b1dd721f44bc432c87c22eb45d0345c4 (127.3.139.66). New cstate: current_term: 1 leader_uuid: "b1dd721f44bc432c87c22eb45d0345c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:12.225509 3672 catalog_manager.cc:5484] T e7049163a60b47c9899ffc9ad679df95 P 0751e317d08443a18b12c780308f9c12 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0751e317d08443a18b12c780308f9c12 (127.3.139.67). New cstate: current_term: 1 leader_uuid: "0751e317d08443a18b12c780308f9c12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } health_report { overall_health: HEALTHY } } }
I20240502 07:58:12.227288 3670 catalog_manager.cc:5484] T c7f79b16b7f949f0aa891ef70bdf6608 P b1dd721f44bc432c87c22eb45d0345c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b1dd721f44bc432c87c22eb45d0345c4 (127.3.139.66). New cstate: current_term: 1 leader_uuid: "b1dd721f44bc432c87c22eb45d0345c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:12.231631 4148 raft_consensus.cc:482] T 94701214c7954905874450dcfa9a678b P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:12.231709 4148 raft_consensus.cc:504] T 94701214c7954905874450dcfa9a678b P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.232389 4148 leader_election.cc:288] T 94701214c7954905874450dcfa9a678b P 76cc6f9adf4c4aceac8d8f7d883deb6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1dd721f44bc432c87c22eb45d0345c4 (127.3.139.66:46179), 0751e317d08443a18b12c780308f9c12 (127.3.139.67:33895)
I20240502 07:58:12.235832 3945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94701214c7954905874450dcfa9a678b" candidate_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1dd721f44bc432c87c22eb45d0345c4" is_pre_election: true
I20240502 07:58:12.235930 3945 raft_consensus.cc:2454] T 94701214c7954905874450dcfa9a678b P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76cc6f9adf4c4aceac8d8f7d883deb6e in term 0.
I20240502 07:58:12.236176 4076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94701214c7954905874450dcfa9a678b" candidate_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0751e317d08443a18b12c780308f9c12" is_pre_election: true
I20240502 07:58:12.236178 3749 leader_election.cc:302] T 94701214c7954905874450dcfa9a678b P 76cc6f9adf4c4aceac8d8f7d883deb6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76cc6f9adf4c4aceac8d8f7d883deb6e, b1dd721f44bc432c87c22eb45d0345c4; no voters:
I20240502 07:58:12.236269 4076 raft_consensus.cc:2454] T 94701214c7954905874450dcfa9a678b P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76cc6f9adf4c4aceac8d8f7d883deb6e in term 0.
I20240502 07:58:12.236351 4148 raft_consensus.cc:2781] T 94701214c7954905874450dcfa9a678b P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:12.236398 4148 raft_consensus.cc:482] T 94701214c7954905874450dcfa9a678b P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:12.236434 4148 raft_consensus.cc:3037] T 94701214c7954905874450dcfa9a678b P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.237011 4148 raft_consensus.cc:504] T 94701214c7954905874450dcfa9a678b P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.237282 4148 leader_election.cc:288] T 94701214c7954905874450dcfa9a678b P 76cc6f9adf4c4aceac8d8f7d883deb6e [CANDIDATE]: Term 1 election: Requested vote from peers b1dd721f44bc432c87c22eb45d0345c4 (127.3.139.66:46179), 0751e317d08443a18b12c780308f9c12 (127.3.139.67:33895)
I20240502 07:58:12.237541 3945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94701214c7954905874450dcfa9a678b" candidate_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1dd721f44bc432c87c22eb45d0345c4"
I20240502 07:58:12.237638 3945 raft_consensus.cc:3037] T 94701214c7954905874450dcfa9a678b P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.237636 4076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94701214c7954905874450dcfa9a678b" candidate_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0751e317d08443a18b12c780308f9c12"
I20240502 07:58:12.237725 4076 raft_consensus.cc:3037] T 94701214c7954905874450dcfa9a678b P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.238194 3945 raft_consensus.cc:2454] T 94701214c7954905874450dcfa9a678b P b1dd721f44bc432c87c22eb45d0345c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76cc6f9adf4c4aceac8d8f7d883deb6e in term 1.
I20240502 07:58:12.238272 4076 raft_consensus.cc:2454] T 94701214c7954905874450dcfa9a678b P 0751e317d08443a18b12c780308f9c12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76cc6f9adf4c4aceac8d8f7d883deb6e in term 1.
I20240502 07:58:12.238572 3749 leader_election.cc:302] T 94701214c7954905874450dcfa9a678b P 76cc6f9adf4c4aceac8d8f7d883deb6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76cc6f9adf4c4aceac8d8f7d883deb6e, b1dd721f44bc432c87c22eb45d0345c4; no voters:
I20240502 07:58:12.238814 4148 raft_consensus.cc:2781] T 94701214c7954905874450dcfa9a678b P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:12.238900 4148 raft_consensus.cc:686] T 94701214c7954905874450dcfa9a678b P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 1 LEADER]: Becoming Leader. State: Replica: 76cc6f9adf4c4aceac8d8f7d883deb6e, State: Running, Role: LEADER
I20240502 07:58:12.239161 4148 consensus_queue.cc:237] T 94701214c7954905874450dcfa9a678b P 76cc6f9adf4c4aceac8d8f7d883deb6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.241395 3670 catalog_manager.cc:5484] T 94701214c7954905874450dcfa9a678b P 76cc6f9adf4c4aceac8d8f7d883deb6e reported cstate change: term changed from 0 to 1, leader changed from <none> to 76cc6f9adf4c4aceac8d8f7d883deb6e (127.3.139.65). New cstate: current_term: 1 leader_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:12.243973 4204 raft_consensus.cc:482] T 87abd4c4107e4a4aa7dfdb55ee00b26f P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:12.244042 4204 raft_consensus.cc:504] T 87abd4c4107e4a4aa7dfdb55ee00b26f P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.244277 4204 leader_election.cc:288] T 87abd4c4107e4a4aa7dfdb55ee00b26f P b1dd721f44bc432c87c22eb45d0345c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76cc6f9adf4c4aceac8d8f7d883deb6e (127.3.139.65:35675), 0751e317d08443a18b12c780308f9c12 (127.3.139.67:33895)
I20240502 07:58:12.244513 3813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87abd4c4107e4a4aa7dfdb55ee00b26f" candidate_uuid: "b1dd721f44bc432c87c22eb45d0345c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" is_pre_election: true
I20240502 07:58:12.244629 3813 raft_consensus.cc:2454] T 87abd4c4107e4a4aa7dfdb55ee00b26f P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1dd721f44bc432c87c22eb45d0345c4 in term 0.
I20240502 07:58:12.244621 4076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87abd4c4107e4a4aa7dfdb55ee00b26f" candidate_uuid: "b1dd721f44bc432c87c22eb45d0345c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0751e317d08443a18b12c780308f9c12" is_pre_election: true
I20240502 07:58:12.244705 4076 raft_consensus.cc:2454] T 87abd4c4107e4a4aa7dfdb55ee00b26f P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1dd721f44bc432c87c22eb45d0345c4 in term 0.
I20240502 07:58:12.244889 3880 leader_election.cc:302] T 87abd4c4107e4a4aa7dfdb55ee00b26f P b1dd721f44bc432c87c22eb45d0345c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76cc6f9adf4c4aceac8d8f7d883deb6e, b1dd721f44bc432c87c22eb45d0345c4; no voters:
I20240502 07:58:12.245024 4204 raft_consensus.cc:2781] T 87abd4c4107e4a4aa7dfdb55ee00b26f P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:12.245077 4204 raft_consensus.cc:482] T 87abd4c4107e4a4aa7dfdb55ee00b26f P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:12.245121 4204 raft_consensus.cc:3037] T 87abd4c4107e4a4aa7dfdb55ee00b26f P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.245661 4204 raft_consensus.cc:504] T 87abd4c4107e4a4aa7dfdb55ee00b26f P b1dd721f44bc432c87c22eb45d0345c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.245930 4204 leader_election.cc:288] T 87abd4c4107e4a4aa7dfdb55ee00b26f P b1dd721f44bc432c87c22eb45d0345c4 [CANDIDATE]: Term 1 election: Requested vote from peers 76cc6f9adf4c4aceac8d8f7d883deb6e (127.3.139.65:35675), 0751e317d08443a18b12c780308f9c12 (127.3.139.67:33895)
I20240502 07:58:12.246186 3813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87abd4c4107e4a4aa7dfdb55ee00b26f" candidate_uuid: "b1dd721f44bc432c87c22eb45d0345c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e"
I20240502 07:58:12.246237 4076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87abd4c4107e4a4aa7dfdb55ee00b26f" candidate_uuid: "b1dd721f44bc432c87c22eb45d0345c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0751e317d08443a18b12c780308f9c12"
I20240502 07:58:12.246284 3813 raft_consensus.cc:3037] T 87abd4c4107e4a4aa7dfdb55ee00b26f P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.246322 4076 raft_consensus.cc:3037] T 87abd4c4107e4a4aa7dfdb55ee00b26f P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.247004 4076 raft_consensus.cc:2454] T 87abd4c4107e4a4aa7dfdb55ee00b26f P 0751e317d08443a18b12c780308f9c12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1dd721f44bc432c87c22eb45d0345c4 in term 1.
I20240502 07:58:12.247004 3813 raft_consensus.cc:2454] T 87abd4c4107e4a4aa7dfdb55ee00b26f P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1dd721f44bc432c87c22eb45d0345c4 in term 1.
I20240502 07:58:12.247274 3880 leader_election.cc:302] T 87abd4c4107e4a4aa7dfdb55ee00b26f P b1dd721f44bc432c87c22eb45d0345c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0751e317d08443a18b12c780308f9c12, b1dd721f44bc432c87c22eb45d0345c4; no voters:
I20240502 07:58:12.247414 4204 raft_consensus.cc:2781] T 87abd4c4107e4a4aa7dfdb55ee00b26f P b1dd721f44bc432c87c22eb45d0345c4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:12.247514 4204 raft_consensus.cc:686] T 87abd4c4107e4a4aa7dfdb55ee00b26f P b1dd721f44bc432c87c22eb45d0345c4 [term 1 LEADER]: Becoming Leader. State: Replica: b1dd721f44bc432c87c22eb45d0345c4, State: Running, Role: LEADER
I20240502 07:58:12.247625 4204 consensus_queue.cc:237] T 87abd4c4107e4a4aa7dfdb55ee00b26f P b1dd721f44bc432c87c22eb45d0345c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.248791 3670 catalog_manager.cc:5484] T 87abd4c4107e4a4aa7dfdb55ee00b26f P b1dd721f44bc432c87c22eb45d0345c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b1dd721f44bc432c87c22eb45d0345c4 (127.3.139.66). New cstate: current_term: 1 leader_uuid: "b1dd721f44bc432c87c22eb45d0345c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:12.278481 4204 raft_consensus.cc:482] T e2bf12199a744839a7e40531e53d3aea P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:12.278554 4204 raft_consensus.cc:504] T e2bf12199a744839a7e40531e53d3aea P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.278812 4204 leader_election.cc:288] T e2bf12199a744839a7e40531e53d3aea P b1dd721f44bc432c87c22eb45d0345c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76cc6f9adf4c4aceac8d8f7d883deb6e (127.3.139.65:35675), 0751e317d08443a18b12c780308f9c12 (127.3.139.67:33895)
I20240502 07:58:12.279116 3813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2bf12199a744839a7e40531e53d3aea" candidate_uuid: "b1dd721f44bc432c87c22eb45d0345c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" is_pre_election: true
I20240502 07:58:12.279143 4076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2bf12199a744839a7e40531e53d3aea" candidate_uuid: "b1dd721f44bc432c87c22eb45d0345c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0751e317d08443a18b12c780308f9c12" is_pre_election: true
I20240502 07:58:12.279228 4076 raft_consensus.cc:2454] T e2bf12199a744839a7e40531e53d3aea P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1dd721f44bc432c87c22eb45d0345c4 in term 0.
I20240502 07:58:12.279228 3813 raft_consensus.cc:2454] T e2bf12199a744839a7e40531e53d3aea P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1dd721f44bc432c87c22eb45d0345c4 in term 0.
I20240502 07:58:12.279443 3880 leader_election.cc:302] T e2bf12199a744839a7e40531e53d3aea P b1dd721f44bc432c87c22eb45d0345c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0751e317d08443a18b12c780308f9c12, b1dd721f44bc432c87c22eb45d0345c4; no voters:
I20240502 07:58:12.279584 4204 raft_consensus.cc:2781] T e2bf12199a744839a7e40531e53d3aea P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:12.279639 4204 raft_consensus.cc:482] T e2bf12199a744839a7e40531e53d3aea P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:12.279665 4204 raft_consensus.cc:3037] T e2bf12199a744839a7e40531e53d3aea P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.280239 4204 raft_consensus.cc:504] T e2bf12199a744839a7e40531e53d3aea P b1dd721f44bc432c87c22eb45d0345c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.280509 4204 leader_election.cc:288] T e2bf12199a744839a7e40531e53d3aea P b1dd721f44bc432c87c22eb45d0345c4 [CANDIDATE]: Term 1 election: Requested vote from peers 76cc6f9adf4c4aceac8d8f7d883deb6e (127.3.139.65:35675), 0751e317d08443a18b12c780308f9c12 (127.3.139.67:33895)
I20240502 07:58:12.280788 4076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2bf12199a744839a7e40531e53d3aea" candidate_uuid: "b1dd721f44bc432c87c22eb45d0345c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0751e317d08443a18b12c780308f9c12"
I20240502 07:58:12.280784 3813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2bf12199a744839a7e40531e53d3aea" candidate_uuid: "b1dd721f44bc432c87c22eb45d0345c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e"
I20240502 07:58:12.280900 4076 raft_consensus.cc:3037] T e2bf12199a744839a7e40531e53d3aea P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.280900 3813 raft_consensus.cc:3037] T e2bf12199a744839a7e40531e53d3aea P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.281598 3813 raft_consensus.cc:2454] T e2bf12199a744839a7e40531e53d3aea P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1dd721f44bc432c87c22eb45d0345c4 in term 1.
I20240502 07:58:12.281598 4076 raft_consensus.cc:2454] T e2bf12199a744839a7e40531e53d3aea P 0751e317d08443a18b12c780308f9c12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1dd721f44bc432c87c22eb45d0345c4 in term 1.
I20240502 07:58:12.281853 3880 leader_election.cc:302] T e2bf12199a744839a7e40531e53d3aea P b1dd721f44bc432c87c22eb45d0345c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76cc6f9adf4c4aceac8d8f7d883deb6e, b1dd721f44bc432c87c22eb45d0345c4; no voters:
I20240502 07:58:12.281991 4204 raft_consensus.cc:2781] T e2bf12199a744839a7e40531e53d3aea P b1dd721f44bc432c87c22eb45d0345c4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:12.282092 4204 raft_consensus.cc:686] T e2bf12199a744839a7e40531e53d3aea P b1dd721f44bc432c87c22eb45d0345c4 [term 1 LEADER]: Becoming Leader. State: Replica: b1dd721f44bc432c87c22eb45d0345c4, State: Running, Role: LEADER
I20240502 07:58:12.282199 4204 consensus_queue.cc:237] T e2bf12199a744839a7e40531e53d3aea P b1dd721f44bc432c87c22eb45d0345c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.283439 3670 catalog_manager.cc:5484] T e2bf12199a744839a7e40531e53d3aea P b1dd721f44bc432c87c22eb45d0345c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b1dd721f44bc432c87c22eb45d0345c4 (127.3.139.66). New cstate: current_term: 1 leader_uuid: "b1dd721f44bc432c87c22eb45d0345c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:12.286629 4204 raft_consensus.cc:482] T 11e2dc546eb3466fab164bfcb0bac310 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:12.286695 4204 raft_consensus.cc:504] T 11e2dc546eb3466fab164bfcb0bac310 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } }
I20240502 07:58:12.286978 4204 leader_election.cc:288] T 11e2dc546eb3466fab164bfcb0bac310 P b1dd721f44bc432c87c22eb45d0345c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0751e317d08443a18b12c780308f9c12 (127.3.139.67:33895), 76cc6f9adf4c4aceac8d8f7d883deb6e (127.3.139.65:35675)
I20240502 07:58:12.287312 3813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11e2dc546eb3466fab164bfcb0bac310" candidate_uuid: "b1dd721f44bc432c87c22eb45d0345c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" is_pre_election: true
I20240502 07:58:12.287410 3813 raft_consensus.cc:2454] T 11e2dc546eb3466fab164bfcb0bac310 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1dd721f44bc432c87c22eb45d0345c4 in term 0.
I20240502 07:58:12.287606 3880 leader_election.cc:302] T 11e2dc546eb3466fab164bfcb0bac310 P b1dd721f44bc432c87c22eb45d0345c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76cc6f9adf4c4aceac8d8f7d883deb6e, b1dd721f44bc432c87c22eb45d0345c4; no voters:
I20240502 07:58:12.287734 4204 raft_consensus.cc:2781] T 11e2dc546eb3466fab164bfcb0bac310 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:12.287775 4204 raft_consensus.cc:482] T 11e2dc546eb3466fab164bfcb0bac310 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:12.287799 4204 raft_consensus.cc:3037] T 11e2dc546eb3466fab164bfcb0bac310 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.288324 4204 raft_consensus.cc:504] T 11e2dc546eb3466fab164bfcb0bac310 P b1dd721f44bc432c87c22eb45d0345c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } }
W20240502 07:58:12.288471 4123 tablet.cc:2369] T 02538cc46cb6465d927569ed749c2154 P 0751e317d08443a18b12c780308f9c12: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 07:58:12.288611 4204 leader_election.cc:288] T 11e2dc546eb3466fab164bfcb0bac310 P b1dd721f44bc432c87c22eb45d0345c4 [CANDIDATE]: Term 1 election: Requested vote from peers 0751e317d08443a18b12c780308f9c12 (127.3.139.67:33895), 76cc6f9adf4c4aceac8d8f7d883deb6e (127.3.139.65:35675)
I20240502 07:58:12.288993 3813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11e2dc546eb3466fab164bfcb0bac310" candidate_uuid: "b1dd721f44bc432c87c22eb45d0345c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e"
I20240502 07:58:12.289077 3813 raft_consensus.cc:3037] T 11e2dc546eb3466fab164bfcb0bac310 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.289616 3813 raft_consensus.cc:2454] T 11e2dc546eb3466fab164bfcb0bac310 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1dd721f44bc432c87c22eb45d0345c4 in term 1.
I20240502 07:58:12.289852 3880 leader_election.cc:302] T 11e2dc546eb3466fab164bfcb0bac310 P b1dd721f44bc432c87c22eb45d0345c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76cc6f9adf4c4aceac8d8f7d883deb6e, b1dd721f44bc432c87c22eb45d0345c4; no voters:
I20240502 07:58:12.289988 4204 raft_consensus.cc:2781] T 11e2dc546eb3466fab164bfcb0bac310 P b1dd721f44bc432c87c22eb45d0345c4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:12.290055 4204 raft_consensus.cc:686] T 11e2dc546eb3466fab164bfcb0bac310 P b1dd721f44bc432c87c22eb45d0345c4 [term 1 LEADER]: Becoming Leader. State: Replica: b1dd721f44bc432c87c22eb45d0345c4, State: Running, Role: LEADER
I20240502 07:58:12.290158 4204 consensus_queue.cc:237] T 11e2dc546eb3466fab164bfcb0bac310 P b1dd721f44bc432c87c22eb45d0345c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } }
I20240502 07:58:12.291409 3670 catalog_manager.cc:5484] T 11e2dc546eb3466fab164bfcb0bac310 P b1dd721f44bc432c87c22eb45d0345c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b1dd721f44bc432c87c22eb45d0345c4 (127.3.139.66). New cstate: current_term: 1 leader_uuid: "b1dd721f44bc432c87c22eb45d0345c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } health_report { overall_health: HEALTHY } } }
I20240502 07:58:12.295238 4076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11e2dc546eb3466fab164bfcb0bac310" candidate_uuid: "b1dd721f44bc432c87c22eb45d0345c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0751e317d08443a18b12c780308f9c12" is_pre_election: true
I20240502 07:58:12.295289 4075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11e2dc546eb3466fab164bfcb0bac310" candidate_uuid: "b1dd721f44bc432c87c22eb45d0345c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0751e317d08443a18b12c780308f9c12"
I20240502 07:58:12.295334 4076 raft_consensus.cc:2454] T 11e2dc546eb3466fab164bfcb0bac310 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1dd721f44bc432c87c22eb45d0345c4 in term 0.
I20240502 07:58:12.315434 4148 raft_consensus.cc:482] T 23d6b429a6374ccd9e5e529186fdcea9 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:12.315516 4148 raft_consensus.cc:504] T 23d6b429a6374ccd9e5e529186fdcea9 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.315847 4148 leader_election.cc:288] T 23d6b429a6374ccd9e5e529186fdcea9 P 76cc6f9adf4c4aceac8d8f7d883deb6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1dd721f44bc432c87c22eb45d0345c4 (127.3.139.66:46179), 0751e317d08443a18b12c780308f9c12 (127.3.139.67:33895)
I20240502 07:58:12.316011 4148 raft_consensus.cc:482] T 2a08d03748e94ebaab74067281c8f8a2 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:12.316094 4148 raft_consensus.cc:504] T 2a08d03748e94ebaab74067281c8f8a2 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.316141 3945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23d6b429a6374ccd9e5e529186fdcea9" candidate_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1dd721f44bc432c87c22eb45d0345c4" is_pre_election: true
I20240502 07:58:12.316262 4075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23d6b429a6374ccd9e5e529186fdcea9" candidate_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0751e317d08443a18b12c780308f9c12" is_pre_election: true
I20240502 07:58:12.316311 3945 raft_consensus.cc:2454] T 23d6b429a6374ccd9e5e529186fdcea9 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76cc6f9adf4c4aceac8d8f7d883deb6e in term 0.
I20240502 07:58:12.316357 4075 raft_consensus.cc:2454] T 23d6b429a6374ccd9e5e529186fdcea9 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76cc6f9adf4c4aceac8d8f7d883deb6e in term 0.
I20240502 07:58:12.316409 4148 leader_election.cc:288] T 2a08d03748e94ebaab74067281c8f8a2 P 76cc6f9adf4c4aceac8d8f7d883deb6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1dd721f44bc432c87c22eb45d0345c4 (127.3.139.66:46179), 0751e317d08443a18b12c780308f9c12 (127.3.139.67:33895)
I20240502 07:58:12.316591 3749 leader_election.cc:302] T 23d6b429a6374ccd9e5e529186fdcea9 P 76cc6f9adf4c4aceac8d8f7d883deb6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76cc6f9adf4c4aceac8d8f7d883deb6e, b1dd721f44bc432c87c22eb45d0345c4; no voters:
I20240502 07:58:12.316687 3945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a08d03748e94ebaab74067281c8f8a2" candidate_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1dd721f44bc432c87c22eb45d0345c4" is_pre_election: true
I20240502 07:58:12.316763 4148 raft_consensus.cc:2781] T 23d6b429a6374ccd9e5e529186fdcea9 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:12.316797 3945 raft_consensus.cc:2454] T 2a08d03748e94ebaab74067281c8f8a2 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76cc6f9adf4c4aceac8d8f7d883deb6e in term 0.
I20240502 07:58:12.316812 4148 raft_consensus.cc:482] T 23d6b429a6374ccd9e5e529186fdcea9 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:12.316846 4148 raft_consensus.cc:3037] T 23d6b429a6374ccd9e5e529186fdcea9 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.316983 4075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a08d03748e94ebaab74067281c8f8a2" candidate_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0751e317d08443a18b12c780308f9c12" is_pre_election: true
I20240502 07:58:12.317070 4075 raft_consensus.cc:2454] T 2a08d03748e94ebaab74067281c8f8a2 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76cc6f9adf4c4aceac8d8f7d883deb6e in term 0.
I20240502 07:58:12.317353 3749 leader_election.cc:302] T 2a08d03748e94ebaab74067281c8f8a2 P 76cc6f9adf4c4aceac8d8f7d883deb6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76cc6f9adf4c4aceac8d8f7d883deb6e, b1dd721f44bc432c87c22eb45d0345c4; no voters:
I20240502 07:58:12.317544 4146 raft_consensus.cc:2781] T 2a08d03748e94ebaab74067281c8f8a2 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:12.317595 4146 raft_consensus.cc:482] T 2a08d03748e94ebaab74067281c8f8a2 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:12.317643 4146 raft_consensus.cc:3037] T 2a08d03748e94ebaab74067281c8f8a2 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.317926 4148 raft_consensus.cc:504] T 23d6b429a6374ccd9e5e529186fdcea9 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.318197 4148 leader_election.cc:288] T 23d6b429a6374ccd9e5e529186fdcea9 P 76cc6f9adf4c4aceac8d8f7d883deb6e [CANDIDATE]: Term 1 election: Requested vote from peers b1dd721f44bc432c87c22eb45d0345c4 (127.3.139.66:46179), 0751e317d08443a18b12c780308f9c12 (127.3.139.67:33895)
I20240502 07:58:12.318356 4146 raft_consensus.cc:504] T 2a08d03748e94ebaab74067281c8f8a2 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.318509 3945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23d6b429a6374ccd9e5e529186fdcea9" candidate_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1dd721f44bc432c87c22eb45d0345c4"
I20240502 07:58:12.318518 4075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23d6b429a6374ccd9e5e529186fdcea9" candidate_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0751e317d08443a18b12c780308f9c12"
I20240502 07:58:12.318601 4075 raft_consensus.cc:3037] T 23d6b429a6374ccd9e5e529186fdcea9 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.318603 3945 raft_consensus.cc:3037] T 23d6b429a6374ccd9e5e529186fdcea9 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.318671 4146 leader_election.cc:288] T 2a08d03748e94ebaab74067281c8f8a2 P 76cc6f9adf4c4aceac8d8f7d883deb6e [CANDIDATE]: Term 1 election: Requested vote from peers b1dd721f44bc432c87c22eb45d0345c4 (127.3.139.66:46179), 0751e317d08443a18b12c780308f9c12 (127.3.139.67:33895)
I20240502 07:58:12.319020 3944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a08d03748e94ebaab74067281c8f8a2" candidate_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1dd721f44bc432c87c22eb45d0345c4"
I20240502 07:58:12.319008 4076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a08d03748e94ebaab74067281c8f8a2" candidate_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0751e317d08443a18b12c780308f9c12"
I20240502 07:58:12.319135 3944 raft_consensus.cc:3037] T 2a08d03748e94ebaab74067281c8f8a2 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.319144 4076 raft_consensus.cc:3037] T 2a08d03748e94ebaab74067281c8f8a2 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.319408 4075 raft_consensus.cc:2454] T 23d6b429a6374ccd9e5e529186fdcea9 P 0751e317d08443a18b12c780308f9c12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76cc6f9adf4c4aceac8d8f7d883deb6e in term 1.
I20240502 07:58:12.319435 3945 raft_consensus.cc:2454] T 23d6b429a6374ccd9e5e529186fdcea9 P b1dd721f44bc432c87c22eb45d0345c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76cc6f9adf4c4aceac8d8f7d883deb6e in term 1.
I20240502 07:58:12.319839 4076 raft_consensus.cc:2454] T 2a08d03748e94ebaab74067281c8f8a2 P 0751e317d08443a18b12c780308f9c12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76cc6f9adf4c4aceac8d8f7d883deb6e in term 1.
I20240502 07:58:12.319967 3944 raft_consensus.cc:2454] T 2a08d03748e94ebaab74067281c8f8a2 P b1dd721f44bc432c87c22eb45d0345c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76cc6f9adf4c4aceac8d8f7d883deb6e in term 1.
I20240502 07:58:12.320183 3749 leader_election.cc:302] T 23d6b429a6374ccd9e5e529186fdcea9 P 76cc6f9adf4c4aceac8d8f7d883deb6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0751e317d08443a18b12c780308f9c12, 76cc6f9adf4c4aceac8d8f7d883deb6e; no voters:
I20240502 07:58:12.320317 4146 raft_consensus.cc:2781] T 23d6b429a6374ccd9e5e529186fdcea9 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:12.320323 3749 leader_election.cc:302] T 2a08d03748e94ebaab74067281c8f8a2 P 76cc6f9adf4c4aceac8d8f7d883deb6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0751e317d08443a18b12c780308f9c12, 76cc6f9adf4c4aceac8d8f7d883deb6e; no voters:
I20240502 07:58:12.320418 4146 raft_consensus.cc:686] T 23d6b429a6374ccd9e5e529186fdcea9 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 1 LEADER]: Becoming Leader. State: Replica: 76cc6f9adf4c4aceac8d8f7d883deb6e, State: Running, Role: LEADER
I20240502 07:58:12.320536 4146 consensus_queue.cc:237] T 23d6b429a6374ccd9e5e529186fdcea9 P 76cc6f9adf4c4aceac8d8f7d883deb6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.320753 4214 raft_consensus.cc:2781] T 2a08d03748e94ebaab74067281c8f8a2 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:12.320842 4214 raft_consensus.cc:686] T 2a08d03748e94ebaab74067281c8f8a2 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 1 LEADER]: Becoming Leader. State: Replica: 76cc6f9adf4c4aceac8d8f7d883deb6e, State: Running, Role: LEADER
I20240502 07:58:12.320945 4214 consensus_queue.cc:237] T 2a08d03748e94ebaab74067281c8f8a2 P 76cc6f9adf4c4aceac8d8f7d883deb6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.321954 3670 catalog_manager.cc:5484] T 23d6b429a6374ccd9e5e529186fdcea9 P 76cc6f9adf4c4aceac8d8f7d883deb6e reported cstate change: term changed from 0 to 1, leader changed from <none> to 76cc6f9adf4c4aceac8d8f7d883deb6e (127.3.139.65). New cstate: current_term: 1 leader_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:12.324463 3670 catalog_manager.cc:5484] T 2a08d03748e94ebaab74067281c8f8a2 P 76cc6f9adf4c4aceac8d8f7d883deb6e reported cstate change: term changed from 0 to 1, leader changed from <none> to 76cc6f9adf4c4aceac8d8f7d883deb6e (127.3.139.65). New cstate: current_term: 1 leader_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:12.356417 4214 raft_consensus.cc:482] T 86519f378c294fca8232a94970dd5339 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:12.356503 4214 raft_consensus.cc:504] T 86519f378c294fca8232a94970dd5339 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.356799 4214 leader_election.cc:288] T 86519f378c294fca8232a94970dd5339 P 76cc6f9adf4c4aceac8d8f7d883deb6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1dd721f44bc432c87c22eb45d0345c4 (127.3.139.66:46179), 0751e317d08443a18b12c780308f9c12 (127.3.139.67:33895)
I20240502 07:58:12.357112 3944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86519f378c294fca8232a94970dd5339" candidate_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1dd721f44bc432c87c22eb45d0345c4" is_pre_election: true
I20240502 07:58:12.357161 4076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86519f378c294fca8232a94970dd5339" candidate_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0751e317d08443a18b12c780308f9c12" is_pre_election: true
I20240502 07:58:12.357225 3944 raft_consensus.cc:2454] T 86519f378c294fca8232a94970dd5339 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76cc6f9adf4c4aceac8d8f7d883deb6e in term 0.
I20240502 07:58:12.357259 4076 raft_consensus.cc:2454] T 86519f378c294fca8232a94970dd5339 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76cc6f9adf4c4aceac8d8f7d883deb6e in term 0.
I20240502 07:58:12.357477 3749 leader_election.cc:302] T 86519f378c294fca8232a94970dd5339 P 76cc6f9adf4c4aceac8d8f7d883deb6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0751e317d08443a18b12c780308f9c12, 76cc6f9adf4c4aceac8d8f7d883deb6e; no voters:
I20240502 07:58:12.357630 4214 raft_consensus.cc:2781] T 86519f378c294fca8232a94970dd5339 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:12.357685 4214 raft_consensus.cc:482] T 86519f378c294fca8232a94970dd5339 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:12.357712 4214 raft_consensus.cc:3037] T 86519f378c294fca8232a94970dd5339 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.358399 4214 raft_consensus.cc:504] T 86519f378c294fca8232a94970dd5339 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.358620 4151 raft_consensus.cc:482] T 9b5aa286c46445a381e581dff1798ed8 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:12.358695 4214 leader_election.cc:288] T 86519f378c294fca8232a94970dd5339 P 76cc6f9adf4c4aceac8d8f7d883deb6e [CANDIDATE]: Term 1 election: Requested vote from peers b1dd721f44bc432c87c22eb45d0345c4 (127.3.139.66:46179), 0751e317d08443a18b12c780308f9c12 (127.3.139.67:33895)
I20240502 07:58:12.358703 4151 raft_consensus.cc:504] T 9b5aa286c46445a381e581dff1798ed8 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.359021 4076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86519f378c294fca8232a94970dd5339" candidate_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0751e317d08443a18b12c780308f9c12"
I20240502 07:58:12.359023 3944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86519f378c294fca8232a94970dd5339" candidate_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1dd721f44bc432c87c22eb45d0345c4"
I20240502 07:58:12.359092 4151 leader_election.cc:288] T 9b5aa286c46445a381e581dff1798ed8 P 0751e317d08443a18b12c780308f9c12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1dd721f44bc432c87c22eb45d0345c4 (127.3.139.66:46179), 76cc6f9adf4c4aceac8d8f7d883deb6e (127.3.139.65:35675)
I20240502 07:58:12.359136 3944 raft_consensus.cc:3037] T 86519f378c294fca8232a94970dd5339 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.359138 4076 raft_consensus.cc:3037] T 86519f378c294fca8232a94970dd5339 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.359448 3813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b5aa286c46445a381e581dff1798ed8" candidate_uuid: "0751e317d08443a18b12c780308f9c12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" is_pre_election: true
I20240502 07:58:12.359556 3813 raft_consensus.cc:2454] T 9b5aa286c46445a381e581dff1798ed8 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0751e317d08443a18b12c780308f9c12 in term 0.
I20240502 07:58:12.359745 3945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b5aa286c46445a381e581dff1798ed8" candidate_uuid: "0751e317d08443a18b12c780308f9c12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1dd721f44bc432c87c22eb45d0345c4" is_pre_election: true
I20240502 07:58:12.359840 3945 raft_consensus.cc:2454] T 9b5aa286c46445a381e581dff1798ed8 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0751e317d08443a18b12c780308f9c12 in term 0.
I20240502 07:58:12.359835 4011 leader_election.cc:302] T 9b5aa286c46445a381e581dff1798ed8 P 0751e317d08443a18b12c780308f9c12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0751e317d08443a18b12c780308f9c12, 76cc6f9adf4c4aceac8d8f7d883deb6e; no voters:
I20240502 07:58:12.359922 4076 raft_consensus.cc:2454] T 86519f378c294fca8232a94970dd5339 P 0751e317d08443a18b12c780308f9c12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76cc6f9adf4c4aceac8d8f7d883deb6e in term 1.
I20240502 07:58:12.359975 3944 raft_consensus.cc:2454] T 86519f378c294fca8232a94970dd5339 P b1dd721f44bc432c87c22eb45d0345c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76cc6f9adf4c4aceac8d8f7d883deb6e in term 1.
I20240502 07:58:12.359997 4151 raft_consensus.cc:2781] T 9b5aa286c46445a381e581dff1798ed8 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:12.360037 4151 raft_consensus.cc:482] T 9b5aa286c46445a381e581dff1798ed8 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:12.360071 4151 raft_consensus.cc:3037] T 9b5aa286c46445a381e581dff1798ed8 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.360141 3749 leader_election.cc:302] T 86519f378c294fca8232a94970dd5339 P 76cc6f9adf4c4aceac8d8f7d883deb6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0751e317d08443a18b12c780308f9c12, 76cc6f9adf4c4aceac8d8f7d883deb6e; no voters:
I20240502 07:58:12.360359 4214 raft_consensus.cc:2781] T 86519f378c294fca8232a94970dd5339 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:12.360447 4214 raft_consensus.cc:686] T 86519f378c294fca8232a94970dd5339 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 1 LEADER]: Becoming Leader. State: Replica: 76cc6f9adf4c4aceac8d8f7d883deb6e, State: Running, Role: LEADER
I20240502 07:58:12.360569 4214 consensus_queue.cc:237] T 86519f378c294fca8232a94970dd5339 P 76cc6f9adf4c4aceac8d8f7d883deb6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.360913 4151 raft_consensus.cc:504] T 9b5aa286c46445a381e581dff1798ed8 P 0751e317d08443a18b12c780308f9c12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.361215 4151 leader_election.cc:288] T 9b5aa286c46445a381e581dff1798ed8 P 0751e317d08443a18b12c780308f9c12 [CANDIDATE]: Term 1 election: Requested vote from peers b1dd721f44bc432c87c22eb45d0345c4 (127.3.139.66:46179), 76cc6f9adf4c4aceac8d8f7d883deb6e (127.3.139.65:35675)
I20240502 07:58:12.361559 3813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b5aa286c46445a381e581dff1798ed8" candidate_uuid: "0751e317d08443a18b12c780308f9c12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e"
I20240502 07:58:12.361660 3813 raft_consensus.cc:3037] T 9b5aa286c46445a381e581dff1798ed8 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.361677 3944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b5aa286c46445a381e581dff1798ed8" candidate_uuid: "0751e317d08443a18b12c780308f9c12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1dd721f44bc432c87c22eb45d0345c4"
I20240502 07:58:12.361788 3944 raft_consensus.cc:3037] T 9b5aa286c46445a381e581dff1798ed8 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.362274 3813 raft_consensus.cc:2454] T 9b5aa286c46445a381e581dff1798ed8 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0751e317d08443a18b12c780308f9c12 in term 1.
I20240502 07:58:12.362200 3670 catalog_manager.cc:5484] T 86519f378c294fca8232a94970dd5339 P 76cc6f9adf4c4aceac8d8f7d883deb6e reported cstate change: term changed from 0 to 1, leader changed from <none> to 76cc6f9adf4c4aceac8d8f7d883deb6e (127.3.139.65). New cstate: current_term: 1 leader_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:12.362512 4011 leader_election.cc:302] T 9b5aa286c46445a381e581dff1798ed8 P 0751e317d08443a18b12c780308f9c12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0751e317d08443a18b12c780308f9c12, 76cc6f9adf4c4aceac8d8f7d883deb6e; no voters:
I20240502 07:58:12.362517 3944 raft_consensus.cc:2454] T 9b5aa286c46445a381e581dff1798ed8 P b1dd721f44bc432c87c22eb45d0345c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0751e317d08443a18b12c780308f9c12 in term 1.
I20240502 07:58:12.362637 4151 raft_consensus.cc:2781] T 9b5aa286c46445a381e581dff1798ed8 P 0751e317d08443a18b12c780308f9c12 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:12.362715 4151 raft_consensus.cc:686] T 9b5aa286c46445a381e581dff1798ed8 P 0751e317d08443a18b12c780308f9c12 [term 1 LEADER]: Becoming Leader. State: Replica: 0751e317d08443a18b12c780308f9c12, State: Running, Role: LEADER
I20240502 07:58:12.362819 4151 consensus_queue.cc:237] T 9b5aa286c46445a381e581dff1798ed8 P 0751e317d08443a18b12c780308f9c12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.364435 3670 catalog_manager.cc:5484] T 9b5aa286c46445a381e581dff1798ed8 P 0751e317d08443a18b12c780308f9c12 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0751e317d08443a18b12c780308f9c12 (127.3.139.67). New cstate: current_term: 1 leader_uuid: "0751e317d08443a18b12c780308f9c12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } health_report { overall_health: HEALTHY } } }
I20240502 07:58:12.369179 4214 raft_consensus.cc:482] T 02538cc46cb6465d927569ed749c2154 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:12.369248 4214 raft_consensus.cc:504] T 02538cc46cb6465d927569ed749c2154 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.369501 4214 leader_election.cc:288] T 02538cc46cb6465d927569ed749c2154 P 76cc6f9adf4c4aceac8d8f7d883deb6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1dd721f44bc432c87c22eb45d0345c4 (127.3.139.66:46179), 0751e317d08443a18b12c780308f9c12 (127.3.139.67:33895)
I20240502 07:58:12.369757 3944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02538cc46cb6465d927569ed749c2154" candidate_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1dd721f44bc432c87c22eb45d0345c4" is_pre_election: true
I20240502 07:58:12.369820 4076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02538cc46cb6465d927569ed749c2154" candidate_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0751e317d08443a18b12c780308f9c12" is_pre_election: true
I20240502 07:58:12.369865 3944 raft_consensus.cc:2454] T 02538cc46cb6465d927569ed749c2154 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76cc6f9adf4c4aceac8d8f7d883deb6e in term 0.
I20240502 07:58:12.369925 4076 raft_consensus.cc:2454] T 02538cc46cb6465d927569ed749c2154 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76cc6f9adf4c4aceac8d8f7d883deb6e in term 0.
I20240502 07:58:12.370092 3749 leader_election.cc:302] T 02538cc46cb6465d927569ed749c2154 P 76cc6f9adf4c4aceac8d8f7d883deb6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76cc6f9adf4c4aceac8d8f7d883deb6e, b1dd721f44bc432c87c22eb45d0345c4; no voters:
I20240502 07:58:12.370250 4214 raft_consensus.cc:2781] T 02538cc46cb6465d927569ed749c2154 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:12.370299 4214 raft_consensus.cc:482] T 02538cc46cb6465d927569ed749c2154 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:12.370342 4214 raft_consensus.cc:3037] T 02538cc46cb6465d927569ed749c2154 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.370916 4214 raft_consensus.cc:504] T 02538cc46cb6465d927569ed749c2154 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.371203 4214 leader_election.cc:288] T 02538cc46cb6465d927569ed749c2154 P 76cc6f9adf4c4aceac8d8f7d883deb6e [CANDIDATE]: Term 1 election: Requested vote from peers b1dd721f44bc432c87c22eb45d0345c4 (127.3.139.66:46179), 0751e317d08443a18b12c780308f9c12 (127.3.139.67:33895)
I20240502 07:58:12.371459 3944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02538cc46cb6465d927569ed749c2154" candidate_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1dd721f44bc432c87c22eb45d0345c4"
I20240502 07:58:12.371505 4076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02538cc46cb6465d927569ed749c2154" candidate_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0751e317d08443a18b12c780308f9c12"
I20240502 07:58:12.371548 3944 raft_consensus.cc:3037] T 02538cc46cb6465d927569ed749c2154 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.371579 4076 raft_consensus.cc:3037] T 02538cc46cb6465d927569ed749c2154 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.372112 3944 raft_consensus.cc:2454] T 02538cc46cb6465d927569ed749c2154 P b1dd721f44bc432c87c22eb45d0345c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76cc6f9adf4c4aceac8d8f7d883deb6e in term 1.
I20240502 07:58:12.372117 4076 raft_consensus.cc:2454] T 02538cc46cb6465d927569ed749c2154 P 0751e317d08443a18b12c780308f9c12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76cc6f9adf4c4aceac8d8f7d883deb6e in term 1.
I20240502 07:58:12.372342 3749 leader_election.cc:302] T 02538cc46cb6465d927569ed749c2154 P 76cc6f9adf4c4aceac8d8f7d883deb6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76cc6f9adf4c4aceac8d8f7d883deb6e, b1dd721f44bc432c87c22eb45d0345c4; no voters:
I20240502 07:58:12.372493 4214 raft_consensus.cc:2781] T 02538cc46cb6465d927569ed749c2154 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:12.372573 4214 raft_consensus.cc:686] T 02538cc46cb6465d927569ed749c2154 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 1 LEADER]: Becoming Leader. State: Replica: 76cc6f9adf4c4aceac8d8f7d883deb6e, State: Running, Role: LEADER
I20240502 07:58:12.372679 4214 consensus_queue.cc:237] T 02538cc46cb6465d927569ed749c2154 P 76cc6f9adf4c4aceac8d8f7d883deb6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.373962 3670 catalog_manager.cc:5484] T 02538cc46cb6465d927569ed749c2154 P 76cc6f9adf4c4aceac8d8f7d883deb6e reported cstate change: term changed from 0 to 1, leader changed from <none> to 76cc6f9adf4c4aceac8d8f7d883deb6e (127.3.139.65). New cstate: current_term: 1 leader_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:12.400377 4214 raft_consensus.cc:482] T 45fa74d198214dc78d63ce787c1490d6 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:12.400449 4214 raft_consensus.cc:504] T 45fa74d198214dc78d63ce787c1490d6 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.400707 4214 leader_election.cc:288] T 45fa74d198214dc78d63ce787c1490d6 P 76cc6f9adf4c4aceac8d8f7d883deb6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1dd721f44bc432c87c22eb45d0345c4 (127.3.139.66:46179), 0751e317d08443a18b12c780308f9c12 (127.3.139.67:33895)
I20240502 07:58:12.400991 3944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45fa74d198214dc78d63ce787c1490d6" candidate_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1dd721f44bc432c87c22eb45d0345c4" is_pre_election: true
I20240502 07:58:12.401042 4076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45fa74d198214dc78d63ce787c1490d6" candidate_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0751e317d08443a18b12c780308f9c12" is_pre_election: true
I20240502 07:58:12.401088 3944 raft_consensus.cc:2454] T 45fa74d198214dc78d63ce787c1490d6 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76cc6f9adf4c4aceac8d8f7d883deb6e in term 0.
I20240502 07:58:12.401140 4076 raft_consensus.cc:2454] T 45fa74d198214dc78d63ce787c1490d6 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76cc6f9adf4c4aceac8d8f7d883deb6e in term 0.
I20240502 07:58:12.401363 3749 leader_election.cc:302] T 45fa74d198214dc78d63ce787c1490d6 P 76cc6f9adf4c4aceac8d8f7d883deb6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76cc6f9adf4c4aceac8d8f7d883deb6e, b1dd721f44bc432c87c22eb45d0345c4; no voters:
I20240502 07:58:12.401516 4214 raft_consensus.cc:2781] T 45fa74d198214dc78d63ce787c1490d6 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:12.401564 4214 raft_consensus.cc:482] T 45fa74d198214dc78d63ce787c1490d6 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:12.401607 4214 raft_consensus.cc:3037] T 45fa74d198214dc78d63ce787c1490d6 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.402217 4214 raft_consensus.cc:504] T 45fa74d198214dc78d63ce787c1490d6 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.402484 4214 leader_election.cc:288] T 45fa74d198214dc78d63ce787c1490d6 P 76cc6f9adf4c4aceac8d8f7d883deb6e [CANDIDATE]: Term 1 election: Requested vote from peers b1dd721f44bc432c87c22eb45d0345c4 (127.3.139.66:46179), 0751e317d08443a18b12c780308f9c12 (127.3.139.67:33895)
I20240502 07:58:12.402741 3944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45fa74d198214dc78d63ce787c1490d6" candidate_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1dd721f44bc432c87c22eb45d0345c4"
I20240502 07:58:12.402789 4076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45fa74d198214dc78d63ce787c1490d6" candidate_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0751e317d08443a18b12c780308f9c12"
I20240502 07:58:12.402828 3944 raft_consensus.cc:3037] T 45fa74d198214dc78d63ce787c1490d6 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.402875 4076 raft_consensus.cc:3037] T 45fa74d198214dc78d63ce787c1490d6 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.403476 3944 raft_consensus.cc:2454] T 45fa74d198214dc78d63ce787c1490d6 P b1dd721f44bc432c87c22eb45d0345c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76cc6f9adf4c4aceac8d8f7d883deb6e in term 1.
I20240502 07:58:12.403512 4076 raft_consensus.cc:2454] T 45fa74d198214dc78d63ce787c1490d6 P 0751e317d08443a18b12c780308f9c12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76cc6f9adf4c4aceac8d8f7d883deb6e in term 1.
I20240502 07:58:12.403725 3749 leader_election.cc:302] T 45fa74d198214dc78d63ce787c1490d6 P 76cc6f9adf4c4aceac8d8f7d883deb6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76cc6f9adf4c4aceac8d8f7d883deb6e, b1dd721f44bc432c87c22eb45d0345c4; no voters:
I20240502 07:58:12.403864 4214 raft_consensus.cc:2781] T 45fa74d198214dc78d63ce787c1490d6 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:12.403954 4214 raft_consensus.cc:686] T 45fa74d198214dc78d63ce787c1490d6 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 1 LEADER]: Becoming Leader. State: Replica: 76cc6f9adf4c4aceac8d8f7d883deb6e, State: Running, Role: LEADER
I20240502 07:58:12.404064 4214 consensus_queue.cc:237] T 45fa74d198214dc78d63ce787c1490d6 P 76cc6f9adf4c4aceac8d8f7d883deb6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.405354 3670 catalog_manager.cc:5484] T 45fa74d198214dc78d63ce787c1490d6 P 76cc6f9adf4c4aceac8d8f7d883deb6e reported cstate change: term changed from 0 to 1, leader changed from <none> to 76cc6f9adf4c4aceac8d8f7d883deb6e (127.3.139.65). New cstate: current_term: 1 leader_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:12.435508 4214 raft_consensus.cc:482] T a3bb326555d747479c66254d5c592b57 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:12.435582 4214 raft_consensus.cc:504] T a3bb326555d747479c66254d5c592b57 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.435864 4214 leader_election.cc:288] T a3bb326555d747479c66254d5c592b57 P 76cc6f9adf4c4aceac8d8f7d883deb6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1dd721f44bc432c87c22eb45d0345c4 (127.3.139.66:46179), 0751e317d08443a18b12c780308f9c12 (127.3.139.67:33895)
I20240502 07:58:12.436151 3944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3bb326555d747479c66254d5c592b57" candidate_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1dd721f44bc432c87c22eb45d0345c4" is_pre_election: true
I20240502 07:58:12.436193 4076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3bb326555d747479c66254d5c592b57" candidate_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0751e317d08443a18b12c780308f9c12" is_pre_election: true
I20240502 07:58:12.436244 3944 raft_consensus.cc:2454] T a3bb326555d747479c66254d5c592b57 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76cc6f9adf4c4aceac8d8f7d883deb6e in term 0.
I20240502 07:58:12.436285 4076 raft_consensus.cc:2454] T a3bb326555d747479c66254d5c592b57 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76cc6f9adf4c4aceac8d8f7d883deb6e in term 0.
I20240502 07:58:12.436460 3749 leader_election.cc:302] T a3bb326555d747479c66254d5c592b57 P 76cc6f9adf4c4aceac8d8f7d883deb6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76cc6f9adf4c4aceac8d8f7d883deb6e, b1dd721f44bc432c87c22eb45d0345c4; no voters:
I20240502 07:58:12.436604 4214 raft_consensus.cc:2781] T a3bb326555d747479c66254d5c592b57 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:12.436652 4214 raft_consensus.cc:482] T a3bb326555d747479c66254d5c592b57 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:12.436687 4214 raft_consensus.cc:3037] T a3bb326555d747479c66254d5c592b57 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.437273 4214 raft_consensus.cc:504] T a3bb326555d747479c66254d5c592b57 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.437534 4214 leader_election.cc:288] T a3bb326555d747479c66254d5c592b57 P 76cc6f9adf4c4aceac8d8f7d883deb6e [CANDIDATE]: Term 1 election: Requested vote from peers b1dd721f44bc432c87c22eb45d0345c4 (127.3.139.66:46179), 0751e317d08443a18b12c780308f9c12 (127.3.139.67:33895)
I20240502 07:58:12.437793 3944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3bb326555d747479c66254d5c592b57" candidate_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1dd721f44bc432c87c22eb45d0345c4"
I20240502 07:58:12.437839 4076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3bb326555d747479c66254d5c592b57" candidate_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0751e317d08443a18b12c780308f9c12"
I20240502 07:58:12.437894 3944 raft_consensus.cc:3037] T a3bb326555d747479c66254d5c592b57 P b1dd721f44bc432c87c22eb45d0345c4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.437911 4076 raft_consensus.cc:3037] T a3bb326555d747479c66254d5c592b57 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.438447 4076 raft_consensus.cc:2454] T a3bb326555d747479c66254d5c592b57 P 0751e317d08443a18b12c780308f9c12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76cc6f9adf4c4aceac8d8f7d883deb6e in term 1.
I20240502 07:58:12.438452 3944 raft_consensus.cc:2454] T a3bb326555d747479c66254d5c592b57 P b1dd721f44bc432c87c22eb45d0345c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76cc6f9adf4c4aceac8d8f7d883deb6e in term 1.
I20240502 07:58:12.438657 3749 leader_election.cc:302] T a3bb326555d747479c66254d5c592b57 P 76cc6f9adf4c4aceac8d8f7d883deb6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76cc6f9adf4c4aceac8d8f7d883deb6e, b1dd721f44bc432c87c22eb45d0345c4; no voters:
I20240502 07:58:12.438813 4214 raft_consensus.cc:2781] T a3bb326555d747479c66254d5c592b57 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:12.438900 4214 raft_consensus.cc:686] T a3bb326555d747479c66254d5c592b57 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 1 LEADER]: Becoming Leader. State: Replica: 76cc6f9adf4c4aceac8d8f7d883deb6e, State: Running, Role: LEADER
I20240502 07:58:12.439014 4214 consensus_queue.cc:237] T a3bb326555d747479c66254d5c592b57 P 76cc6f9adf4c4aceac8d8f7d883deb6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } }
I20240502 07:58:12.440312 3670 catalog_manager.cc:5484] T a3bb326555d747479c66254d5c592b57 P 76cc6f9adf4c4aceac8d8f7d883deb6e reported cstate change: term changed from 0 to 1, leader changed from <none> to 76cc6f9adf4c4aceac8d8f7d883deb6e (127.3.139.65). New cstate: current_term: 1 leader_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 } health_report { overall_health: UNKNOWN } } }
W20240502 07:58:12.546787 4222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:12.546989 4222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:12.549957 4222 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_0'
I20240502 07:58:12.619729 3813 raft_consensus.cc:3037] T e7049163a60b47c9899ffc9ad679df95 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.621398 4151 consensus_queue.cc:1035] T e7049163a60b47c9899ffc9ad679df95 P 0751e317d08443a18b12c780308f9c12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:12.628350 4196 consensus_queue.cc:1035] T e7049163a60b47c9899ffc9ad679df95 P 0751e317d08443a18b12c780308f9c12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:12.633404 3813 raft_consensus.cc:1261] T 9b5aa286c46445a381e581dff1798ed8 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 1 FOLLOWER]: Refusing update from remote peer 0751e317d08443a18b12c780308f9c12: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:12.634054 3945 raft_consensus.cc:1261] T 9b5aa286c46445a381e581dff1798ed8 P b1dd721f44bc432c87c22eb45d0345c4 [term 1 FOLLOWER]: Refusing update from remote peer 0751e317d08443a18b12c780308f9c12: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:12.634327 3943 raft_consensus.cc:1261] T b99f9c244e014973a024cb91904a64f8 P b1dd721f44bc432c87c22eb45d0345c4 [term 1 FOLLOWER]: Refusing update from remote peer 0751e317d08443a18b12c780308f9c12: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:12.636269 4151 consensus_queue.cc:1035] T 9b5aa286c46445a381e581dff1798ed8 P 0751e317d08443a18b12c780308f9c12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:12.636915 4196 consensus_queue.cc:1035] T 9b5aa286c46445a381e581dff1798ed8 P 0751e317d08443a18b12c780308f9c12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:12.637161 4196 consensus_queue.cc:1035] T b99f9c244e014973a024cb91904a64f8 P 0751e317d08443a18b12c780308f9c12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:12.637485 3813 raft_consensus.cc:1261] T b99f9c244e014973a024cb91904a64f8 P 76cc6f9adf4c4aceac8d8f7d883deb6e [term 1 FOLLOWER]: Refusing update from remote peer 0751e317d08443a18b12c780308f9c12: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:12.638072 4076 raft_consensus.cc:1261] T 02538cc46cb6465d927569ed749c2154 P 0751e317d08443a18b12c780308f9c12 [term 1 FOLLOWER]: Refusing update from remote peer 76cc6f9adf4c4aceac8d8f7d883deb6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:12.638113 4151 consensus_queue.cc:1035] T b99f9c244e014973a024cb91904a64f8 P 0751e317d08443a18b12c780308f9c12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:12.638757 4148 consensus_queue.cc:1035] T 02538cc46cb6465d927569ed749c2154 P 76cc6f9adf4c4aceac8d8f7d883deb6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:12.639012 3942 raft_consensus.cc:1261] T 02538cc46cb6465d927569ed749c2154 P b1dd721f44bc432c87c22eb45d0345c4 [term 1 FOLLOWER]: Refusing update from remote peer 76cc6f9adf4c4aceac8d8f7d883deb6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:12.641646 4074 raft_consensus.cc:1261] T 23d6b429a6374ccd9e5e529186fdcea9 P 0751e317d08443a18b12c780308f9c12 [term 1 FOLLOWER]: Refusing update from remote peer 76cc6f9adf4c4aceac8d8f7d883deb6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:12.643357 4148 consensus_queue.cc:1035] T 23d6b429a6374ccd9e5e529186fdcea9 P 76cc6f9adf4c4aceac8d8f7d883deb6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:12.643472 4073 raft_consensus.cc:1261] T 86519f378c294fca8232a94970dd5339 P 0751e317d08443a18b12c780308f9c12 [term 1 FOLLOWER]: Refusing update from remote peer 76cc6f9adf4c4aceac8d8f7d883deb6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:12.643796 4074 raft_consensus.cc:1261] T 94701214c7954905874450dcfa9a678b P 0751e317d08443a18b12c780308f9c12 [term 1 FOLLOWER]: Refusing update from remote peer 76cc6f9adf4c4aceac8d8f7d883deb6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:12.644069 4076 raft_consensus.cc:1261] T 45fa74d198214dc78d63ce787c1490d6 P 0751e317d08443a18b12c780308f9c12 [term 1 FOLLOWER]: Refusing update from remote peer 76cc6f9adf4c4aceac8d8f7d883deb6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:12.644765 3941 raft_consensus.cc:1261] T 23d6b429a6374ccd9e5e529186fdcea9 P b1dd721f44bc432c87c22eb45d0345c4 [term 1 FOLLOWER]: Refusing update from remote peer 76cc6f9adf4c4aceac8d8f7d883deb6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:12.645048 3940 raft_consensus.cc:1261] T 94701214c7954905874450dcfa9a678b P b1dd721f44bc432c87c22eb45d0345c4 [term 1 FOLLOWER]: Refusing update from remote peer 76cc6f9adf4c4aceac8d8f7d883deb6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:12.645267 3939 raft_consensus.cc:1261] T 86519f378c294fca8232a94970dd5339 P b1dd721f44bc432c87c22eb45d0345c4 [term 1 FOLLOWER]: Refusing update from remote peer 76cc6f9adf4c4aceac8d8f7d883deb6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:12.645540 3942 raft_consensus.cc:1261] T 45fa74d198214dc78d63ce787c1490d6 P b1dd721f44bc432c87c22eb45d0345c4 [term 1 FOLLOWER]: Refusing update from remote peer 76cc6f9adf4c4aceac8d8f7d883deb6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:12.647500 4247 mvcc.cc:205] Tried to move back new op lower bound from 7023151893014913024 to 7023151891919548416. Current Snapshot: MvccSnapshot[applied={T|T < 7023151893014913024}]
I20240502 07:58:12.648594 4148 consensus_queue.cc:1035] T 86519f378c294fca8232a94970dd5339 P 76cc6f9adf4c4aceac8d8f7d883deb6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:12.648753 4214 consensus_queue.cc:1035] T 94701214c7954905874450dcfa9a678b P 76cc6f9adf4c4aceac8d8f7d883deb6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:12.648864 4148 consensus_queue.cc:1035] T 45fa74d198214dc78d63ce787c1490d6 P 76cc6f9adf4c4aceac8d8f7d883deb6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:12.649075 4148 consensus_queue.cc:1035] T 02538cc46cb6465d927569ed749c2154 P 76cc6f9adf4c4aceac8d8f7d883deb6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:12.649295 4148 consensus_queue.cc:1035] T 94701214c7954905874450dcfa9a678b P 76cc6f9adf4c4aceac8d8f7d883deb6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:12.649518 4146 consensus_queue.cc:1035] T 86519f378c294fca8232a94970dd5339 P 76cc6f9adf4c4aceac8d8f7d883deb6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:12.649286 4214 consensus_queue.cc:1035] T 23d6b429a6374ccd9e5e529186fdcea9 P 76cc6f9adf4c4aceac8d8f7d883deb6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:12.650720 4202 consensus_queue.cc:1035] T c7f79b16b7f949f0aa891ef70bdf6608 P b1dd721f44bc432c87c22eb45d0345c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:12.651784 4248 mvcc.cc:205] Tried to move back new op lower bound from 7023151893039919104 to 7023151891959570432. Current Snapshot: MvccSnapshot[applied={T|T < 7023151893039919104}]
I20240502 07:58:12.656908 4146 consensus_queue.cc:1035] T 45fa74d198214dc78d63ce787c1490d6 P 76cc6f9adf4c4aceac8d8f7d883deb6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:12.665441 4253 mvcc.cc:205] Tried to move back new op lower bound from 7023151893039919104 to 7023151891959570432. Current Snapshot: MvccSnapshot[applied={T|T < 7023151893039919104}]
I20240502 07:58:12.690251 4204 consensus_queue.cc:1035] T c7f79b16b7f949f0aa891ef70bdf6608 P b1dd721f44bc432c87c22eb45d0345c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:12.691754 4290 consensus_queue.cc:1035] T c501775259824616959fab8bca85a93b P 0751e317d08443a18b12c780308f9c12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:12.697331 4150 consensus_queue.cc:1035] T 87abd4c4107e4a4aa7dfdb55ee00b26f P b1dd721f44bc432c87c22eb45d0345c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:12.718853 4287 consensus_queue.cc:1035] T c501775259824616959fab8bca85a93b P 0751e317d08443a18b12c780308f9c12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 144.556 ms
time per row: 0.0722778 ms
I20240502 07:58:12.732486 4203 consensus_queue.cc:1035] T 87abd4c4107e4a4aa7dfdb55ee00b26f P b1dd721f44bc432c87c22eb45d0345c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:12.734426 4148 consensus_queue.cc:1035] T 2a08d03748e94ebaab74067281c8f8a2 P 76cc6f9adf4c4aceac8d8f7d883deb6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:12.738807 4148 consensus_queue.cc:1035] T 2a08d03748e94ebaab74067281c8f8a2 P 76cc6f9adf4c4aceac8d8f7d883deb6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:12.751816 4202 consensus_queue.cc:1035] T 1839e64af65e4171a1222704f19d1856 P b1dd721f44bc432c87c22eb45d0345c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:12.754448 4150 consensus_queue.cc:1035] T 1839e64af65e4171a1222704f19d1856 P b1dd721f44bc432c87c22eb45d0345c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:12.785815 4203 consensus_queue.cc:1035] T e2bf12199a744839a7e40531e53d3aea P b1dd721f44bc432c87c22eb45d0345c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:12.788246 4203 consensus_queue.cc:1035] T e2bf12199a744839a7e40531e53d3aea P b1dd721f44bc432c87c22eb45d0345c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:12.850504 4076 raft_consensus.cc:3037] T 11e2dc546eb3466fab164bfcb0bac310 P 0751e317d08443a18b12c780308f9c12 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.851751 4203 consensus_queue.cc:1035] T 11e2dc546eb3466fab164bfcb0bac310 P b1dd721f44bc432c87c22eb45d0345c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:12.854851 4150 consensus_queue.cc:1035] T 11e2dc546eb3466fab164bfcb0bac310 P b1dd721f44bc432c87c22eb45d0345c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76cc6f9adf4c4aceac8d8f7d883deb6e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:12.875491 4148 consensus_queue.cc:1035] T a3bb326555d747479c66254d5c592b57 P 76cc6f9adf4c4aceac8d8f7d883deb6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1dd721f44bc432c87c22eb45d0345c4" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:12.878810 4272 consensus_queue.cc:1035] T a3bb326555d747479c66254d5c592b57 P 76cc6f9adf4c4aceac8d8f7d883deb6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0751e317d08443a18b12c780308f9c12" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:12.920938 4056 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:58:12.921391 3915 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:58:12.927166 3792 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:58:13.030790 4056 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:58:13.032439 3915 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:58:13.032766 3792 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:58:13.129402 3792 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:58:13.129398 4056 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:58:13.141752 3915 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:58:13.358480 4056 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:58:13.358748 3915 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:58:13.360554 3792 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:58:13.442678 4056 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:58:13.443769 3915 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:58:13.445036 3792 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:58:13.477623 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 3732
I20240502 07:58:13.483353 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 3863
I20240502 07:58:13.488466 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 3994
I20240502 07:58:13.493325 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 3641
2024-05-02T07:58:13Z chronyd exiting
[ OK ] TableRangeRebalancingTest.Basic (5930 ms)
[----------] 1 test from TableRangeRebalancingTest (5930 ms total)
[----------] 1 test from RebalanceStartSafetyTest
[ RUN ] RebalanceStartSafetyTest.TooManyIgnoredTservers/1
I20240502 07:58:13.519599 3629 test_util.cc:268] Using random seed: -150389525
I20240502 07:58:13.520299 3629 ts_itest-base.cc:112] Starting cluster with:
I20240502 07:58:13.520339 3629 ts_itest-base.cc:113] --------------
I20240502 07:58:13.520354 3629 ts_itest-base.cc:114] 5 tablet servers
I20240502 07:58:13.520366 3629 ts_itest-base.cc:115] 3 replicas per TS
I20240502 07:58:13.520395 3629 ts_itest-base.cc:116] --------------
2024-05-02T07:58:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-02T07:58:13Z Disabled control of system clock
I20240502 07:58:13.527217 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.139.126:39833
--webserver_interface=127.3.139.126
--webserver_port=0
--builtin_ntp_servers=127.3.139.84:42539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.139.126:39833
--raft_prepare_replacement_before_eviction=true
--enable_range_replica_placement=false with env {}
W20240502 07:58:13.624264 4470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:13.624473 4470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:13.624529 4470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:13.627418 4470 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:58:13.627475 4470 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240502 07:58:13.627502 4470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:13.627526 4470 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240502 07:58:13.627558 4470 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240502 07:58:13.630964 4470 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:42539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.3.139.126:39833
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.139.126:39833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.3.139.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.4470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:13.631704 4470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:13.632675 4470 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:13.639391 4478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:13.639696 4476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:13.640110 4470 server_base.cc:1034] running on GCE node
W20240502 07:58:13.640396 4475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:13.640846 4470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:13.641618 4470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:13.642755 4470 hybrid_clock.cc:648] HybridClock initialized: now 1714636693642741 us; error 29 us; skew 500 ppm
I20240502 07:58:13.644445 4470 webserver.cc:460] Webserver started at http://127.3.139.126:40847/ using document root <none> and password file <none>
I20240502 07:58:13.644918 4470 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:13.644969 4470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:13.645130 4470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:13.646585 4470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "134b0226773b451c964f97744fca8ccf"
format_stamp: "Formatted at 2024-05-02 07:58:13 on dist-test-slave-twcr"
I20240502 07:58:13.647003 4470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "134b0226773b451c964f97744fca8ccf"
format_stamp: "Formatted at 2024-05-02 07:58:13 on dist-test-slave-twcr"
I20240502 07:58:13.649871 4470 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20240502 07:58:13.651811 4484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:13.652694 4470 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:13.652793 4470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal
uuid: "134b0226773b451c964f97744fca8ccf"
format_stamp: "Formatted at 2024-05-02 07:58:13 on dist-test-slave-twcr"
I20240502 07:58:13.652866 4470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:13.666330 4470 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:13.666829 4470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:13.666965 4470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:13.672797 4470 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.126:39833
I20240502 07:58:13.672803 4536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.126:39833 every 8 connection(s)
I20240502 07:58:13.673646 4470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240502 07:58:13.676009 4537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:13.680514 4537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 134b0226773b451c964f97744fca8ccf: Bootstrap starting.
I20240502 07:58:13.680936 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 4470
I20240502 07:58:13.681046 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240502 07:58:13.682855 4537 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 134b0226773b451c964f97744fca8ccf: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:13.683498 4537 log.cc:824] T 00000000000000000000000000000000 P 134b0226773b451c964f97744fca8ccf: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:13.684958 4537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 134b0226773b451c964f97744fca8ccf: No bootstrap required, opened a new log
I20240502 07:58:13.687306 4537 raft_consensus.cc:348] T 00000000000000000000000000000000 P 134b0226773b451c964f97744fca8ccf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "134b0226773b451c964f97744fca8ccf" member_type: VOTER last_known_addr { host: "127.3.139.126" port: 39833 } }
I20240502 07:58:13.687477 4537 raft_consensus.cc:374] T 00000000000000000000000000000000 P 134b0226773b451c964f97744fca8ccf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:13.687537 4537 raft_consensus.cc:729] T 00000000000000000000000000000000 P 134b0226773b451c964f97744fca8ccf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 134b0226773b451c964f97744fca8ccf, State: Initialized, Role: FOLLOWER
I20240502 07:58:13.687911 4537 consensus_queue.cc:260] T 00000000000000000000000000000000 P 134b0226773b451c964f97744fca8ccf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "134b0226773b451c964f97744fca8ccf" member_type: VOTER last_known_addr { host: "127.3.139.126" port: 39833 } }
I20240502 07:58:13.688030 4537 raft_consensus.cc:388] T 00000000000000000000000000000000 P 134b0226773b451c964f97744fca8ccf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 07:58:13.688088 4537 raft_consensus.cc:482] T 00000000000000000000000000000000 P 134b0226773b451c964f97744fca8ccf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 07:58:13.688190 4537 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 134b0226773b451c964f97744fca8ccf [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:13.688771 4537 raft_consensus.cc:504] T 00000000000000000000000000000000 P 134b0226773b451c964f97744fca8ccf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "134b0226773b451c964f97744fca8ccf" member_type: VOTER last_known_addr { host: "127.3.139.126" port: 39833 } }
I20240502 07:58:13.689059 4537 leader_election.cc:302] T 00000000000000000000000000000000 P 134b0226773b451c964f97744fca8ccf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 134b0226773b451c964f97744fca8ccf; no voters:
I20240502 07:58:13.689304 4537 leader_election.cc:288] T 00000000000000000000000000000000 P 134b0226773b451c964f97744fca8ccf [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 07:58:13.689399 4542 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 134b0226773b451c964f97744fca8ccf [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:13.689603 4542 raft_consensus.cc:686] T 00000000000000000000000000000000 P 134b0226773b451c964f97744fca8ccf [term 1 LEADER]: Becoming Leader. State: Replica: 134b0226773b451c964f97744fca8ccf, State: Running, Role: LEADER
I20240502 07:58:13.689864 4542 consensus_queue.cc:237] T 00000000000000000000000000000000 P 134b0226773b451c964f97744fca8ccf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "134b0226773b451c964f97744fca8ccf" member_type: VOTER last_known_addr { host: "127.3.139.126" port: 39833 } }
I20240502 07:58:13.690202 4537 sys_catalog.cc:564] T 00000000000000000000000000000000 P 134b0226773b451c964f97744fca8ccf [sys.catalog]: configured and running, proceeding with master startup.
I20240502 07:58:13.691201 4544 sys_catalog.cc:455] T 00000000000000000000000000000000 P 134b0226773b451c964f97744fca8ccf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 134b0226773b451c964f97744fca8ccf. Latest consensus state: current_term: 1 leader_uuid: "134b0226773b451c964f97744fca8ccf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "134b0226773b451c964f97744fca8ccf" member_type: VOTER last_known_addr { host: "127.3.139.126" port: 39833 } } }
I20240502 07:58:13.691197 4543 sys_catalog.cc:455] T 00000000000000000000000000000000 P 134b0226773b451c964f97744fca8ccf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "134b0226773b451c964f97744fca8ccf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "134b0226773b451c964f97744fca8ccf" member_type: VOTER last_known_addr { host: "127.3.139.126" port: 39833 } } }
I20240502 07:58:13.691370 4543 sys_catalog.cc:458] T 00000000000000000000000000000000 P 134b0226773b451c964f97744fca8ccf [sys.catalog]: This master's current role is: LEADER
I20240502 07:58:13.691584 4544 sys_catalog.cc:458] T 00000000000000000000000000000000 P 134b0226773b451c964f97744fca8ccf [sys.catalog]: This master's current role is: LEADER
I20240502 07:58:13.691730 4551 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240502 07:58:13.694022 4551 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240502 07:58:13.698536 4551 catalog_manager.cc:1347] Generated new cluster ID: 1148edceddc3414993ed182f9cf86c50
I20240502 07:58:13.698596 4551 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240502 07:58:13.720472 4551 catalog_manager.cc:1370] Generated new certificate authority record
I20240502 07:58:13.721349 4551 catalog_manager.cc:1504] Loading token signing keys...
I20240502 07:58:13.730023 4551 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 134b0226773b451c964f97744fca8ccf: Generated new TSK 0
I20240502 07:58:13.730681 4551 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240502 07:58:13.735761 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.65:0
--local_ip_for_outbound_sockets=127.3.139.65
--webserver_interface=127.3.139.65
--webserver_port=0
--tserver_master_addrs=127.3.139.126:39833
--builtin_ntp_servers=127.3.139.84:42539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240502 07:58:13.832836 4561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:13.833024 4561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:13.833081 4561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:13.835954 4561 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:58:13.836014 4561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:13.836091 4561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.65
I20240502 07:58:13.839602 4561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:42539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.3.139.65
--webserver_port=0
--tserver_master_addrs=127.3.139.126:39833
--never_fsync=true
--heap_profile_path=/tmp/kudu.4561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.65
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:13.840342 4561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:13.841368 4561 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:13.849031 4569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:13.849360 4567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:13.849800 4566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:13.850314 4561 server_base.cc:1034] running on GCE node
I20240502 07:58:13.850657 4561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:13.851181 4561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:13.852319 4561 hybrid_clock.cc:648] HybridClock initialized: now 1714636693852303 us; error 29 us; skew 500 ppm
I20240502 07:58:13.853955 4561 webserver.cc:460] Webserver started at http://127.3.139.65:36813/ using document root <none> and password file <none>
I20240502 07:58:13.854444 4561 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:13.854498 4561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:13.854655 4561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:13.856228 4561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "3b0cf2effa6144788c6182357264817b"
format_stamp: "Formatted at 2024-05-02 07:58:13 on dist-test-slave-twcr"
I20240502 07:58:13.856674 4561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "3b0cf2effa6144788c6182357264817b"
format_stamp: "Formatted at 2024-05-02 07:58:13 on dist-test-slave-twcr"
I20240502 07:58:13.860000 4561 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 07:58:13.861975 4575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:13.862865 4561 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:13.862967 4561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "3b0cf2effa6144788c6182357264817b"
format_stamp: "Formatted at 2024-05-02 07:58:13 on dist-test-slave-twcr"
I20240502 07:58:13.863039 4561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:13.888350 4561 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:13.888880 4561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:13.889030 4561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:13.889509 4561 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:13.890347 4561 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:13.890398 4561 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:13.890450 4561 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:13.890475 4561 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:13.898480 4561 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.65:35895
I20240502 07:58:13.898514 4688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.65:35895 every 8 connection(s)
I20240502 07:58:13.899317 4561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240502 07:58:13.900666 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 4561
I20240502 07:58:13.900768 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240502 07:58:13.902961 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.66:0
--local_ip_for_outbound_sockets=127.3.139.66
--webserver_interface=127.3.139.66
--webserver_port=0
--tserver_master_addrs=127.3.139.126:39833
--builtin_ntp_servers=127.3.139.84:42539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 07:58:13.908066 4689 heartbeater.cc:347] Connected to a master server at 127.3.139.126:39833
I20240502 07:58:13.908248 4689 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:13.908692 4689 heartbeater.cc:511] Master 127.3.139.126:39833 requested a full tablet report, sending...
I20240502 07:58:13.909950 4501 ts_manager.cc:194] Registered new tserver with Master: 3b0cf2effa6144788c6182357264817b (127.3.139.65:35895)
I20240502 07:58:13.911018 4501 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.65:48489
W20240502 07:58:14.000542 4692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:14.000717 4692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:14.000770 4692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:14.003854 4692 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:58:14.003917 4692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:14.004004 4692 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.66
I20240502 07:58:14.007444 4692 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:42539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.3.139.66
--webserver_port=0
--tserver_master_addrs=127.3.139.126:39833
--never_fsync=true
--heap_profile_path=/tmp/kudu.4692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.66
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:14.008121 4692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:14.009035 4692 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:14.016317 4697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:14.016719 4700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:14.017086 4698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:14.017613 4692 server_base.cc:1034] running on GCE node
I20240502 07:58:14.017923 4692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:14.018380 4692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:14.019522 4692 hybrid_clock.cc:648] HybridClock initialized: now 1714636694019512 us; error 22 us; skew 500 ppm
I20240502 07:58:14.021188 4692 webserver.cc:460] Webserver started at http://127.3.139.66:43033/ using document root <none> and password file <none>
I20240502 07:58:14.021652 4692 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:14.021704 4692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:14.021862 4692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:14.023308 4692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "73cf6b5f71d64fb29b2ee84379fe4e04"
format_stamp: "Formatted at 2024-05-02 07:58:14 on dist-test-slave-twcr"
I20240502 07:58:14.023711 4692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "73cf6b5f71d64fb29b2ee84379fe4e04"
format_stamp: "Formatted at 2024-05-02 07:58:14 on dist-test-slave-twcr"
I20240502 07:58:14.026456 4692 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:14.028309 4706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:14.029140 4692 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:14.029232 4692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "73cf6b5f71d64fb29b2ee84379fe4e04"
format_stamp: "Formatted at 2024-05-02 07:58:14 on dist-test-slave-twcr"
I20240502 07:58:14.029304 4692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:14.066183 4692 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:14.066675 4692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:14.066817 4692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:14.067286 4692 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:14.068073 4692 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:14.068117 4692 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:14.068163 4692 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:14.068190 4692 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:14.075625 4692 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.66:46151
I20240502 07:58:14.075670 4819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.66:46151 every 8 connection(s)
I20240502 07:58:14.076404 4692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240502 07:58:14.078107 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 4692
I20240502 07:58:14.078219 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240502 07:58:14.080322 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.67:0
--local_ip_for_outbound_sockets=127.3.139.67
--webserver_interface=127.3.139.67
--webserver_port=0
--tserver_master_addrs=127.3.139.126:39833
--builtin_ntp_servers=127.3.139.84:42539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 07:58:14.084895 4820 heartbeater.cc:347] Connected to a master server at 127.3.139.126:39833
I20240502 07:58:14.085078 4820 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:14.085497 4820 heartbeater.cc:511] Master 127.3.139.126:39833 requested a full tablet report, sending...
I20240502 07:58:14.086295 4501 ts_manager.cc:194] Registered new tserver with Master: 73cf6b5f71d64fb29b2ee84379fe4e04 (127.3.139.66:46151)
I20240502 07:58:14.086804 4501 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.66:45521
W20240502 07:58:14.176540 4823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:14.176705 4823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:14.176748 4823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:14.179633 4823 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:58:14.179685 4823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:14.179756 4823 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.67
I20240502 07:58:14.183145 4823 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:42539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.3.139.67
--webserver_port=0
--tserver_master_addrs=127.3.139.126:39833
--never_fsync=true
--heap_profile_path=/tmp/kudu.4823
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.67
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:14.183843 4823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:14.184762 4823 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:14.191802 4828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:14.192162 4829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:14.192534 4831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:14.193001 4823 server_base.cc:1034] running on GCE node
I20240502 07:58:14.193313 4823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:14.193765 4823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:14.194906 4823 hybrid_clock.cc:648] HybridClock initialized: now 1714636694194898 us; error 29 us; skew 500 ppm
I20240502 07:58:14.196609 4823 webserver.cc:460] Webserver started at http://127.3.139.67:34111/ using document root <none> and password file <none>
I20240502 07:58:14.197085 4823 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:14.197190 4823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:14.197394 4823 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:14.198849 4823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "bf6bb6c77cac4487bb71e7b2ea1b1690"
format_stamp: "Formatted at 2024-05-02 07:58:14 on dist-test-slave-twcr"
I20240502 07:58:14.199286 4823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "bf6bb6c77cac4487bb71e7b2ea1b1690"
format_stamp: "Formatted at 2024-05-02 07:58:14 on dist-test-slave-twcr"
I20240502 07:58:14.202298 4823 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20240502 07:58:14.204202 4837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:14.205093 4823 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:14.205191 4823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "bf6bb6c77cac4487bb71e7b2ea1b1690"
format_stamp: "Formatted at 2024-05-02 07:58:14 on dist-test-slave-twcr"
I20240502 07:58:14.205261 4823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:14.217114 4823 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:14.217571 4823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:14.217695 4823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:14.218128 4823 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:14.218858 4823 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:14.218902 4823 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:14.218947 4823 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:14.218978 4823 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:14.226500 4823 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.67:41209
I20240502 07:58:14.226541 4950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.67:41209 every 8 connection(s)
I20240502 07:58:14.227280 4823 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240502 07:58:14.234694 4951 heartbeater.cc:347] Connected to a master server at 127.3.139.126:39833
I20240502 07:58:14.234877 4951 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:14.235146 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 4823
I20240502 07:58:14.235232 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240502 07:58:14.235369 4951 heartbeater.cc:511] Master 127.3.139.126:39833 requested a full tablet report, sending...
I20240502 07:58:14.236246 4499 ts_manager.cc:194] Registered new tserver with Master: bf6bb6c77cac4487bb71e7b2ea1b1690 (127.3.139.67:41209)
I20240502 07:58:14.236795 4499 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.67:54639
I20240502 07:58:14.237008 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.68:0
--local_ip_for_outbound_sockets=127.3.139.68
--webserver_interface=127.3.139.68
--webserver_port=0
--tserver_master_addrs=127.3.139.126:39833
--builtin_ntp_servers=127.3.139.84:42539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240502 07:58:14.334054 4954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:14.334224 4954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:14.334266 4954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:14.337249 4954 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:58:14.337316 4954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:14.337388 4954 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.68
I20240502 07:58:14.340785 4954 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:42539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.68:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.3.139.68
--webserver_port=0
--tserver_master_addrs=127.3.139.126:39833
--never_fsync=true
--heap_profile_path=/tmp/kudu.4954
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.68
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:14.341476 4954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:14.342422 4954 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:14.349598 4959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:14.349993 4960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:14.350378 4962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:14.350800 4954 server_base.cc:1034] running on GCE node
I20240502 07:58:14.351150 4954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:14.351629 4954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:14.352763 4954 hybrid_clock.cc:648] HybridClock initialized: now 1714636694352755 us; error 21 us; skew 500 ppm
I20240502 07:58:14.354321 4954 webserver.cc:460] Webserver started at http://127.3.139.68:45095/ using document root <none> and password file <none>
I20240502 07:58:14.354784 4954 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:14.354838 4954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:14.354995 4954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:14.356537 4954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "bca5cc35c80a4c0b9cf5f476b9c3b741"
format_stamp: "Formatted at 2024-05-02 07:58:14 on dist-test-slave-twcr"
I20240502 07:58:14.356961 4954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "bca5cc35c80a4c0b9cf5f476b9c3b741"
format_stamp: "Formatted at 2024-05-02 07:58:14 on dist-test-slave-twcr"
I20240502 07:58:14.359864 4954 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20240502 07:58:14.361774 4968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:14.362629 4954 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20240502 07:58:14.362722 4954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "bca5cc35c80a4c0b9cf5f476b9c3b741"
format_stamp: "Formatted at 2024-05-02 07:58:14 on dist-test-slave-twcr"
I20240502 07:58:14.362792 4954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:14.373346 4954 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:14.373793 4954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:14.373939 4954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:14.374372 4954 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:14.375150 4954 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:14.375195 4954 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:14.375241 4954 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:14.375267 4954 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:14.382531 4954 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.68:39521
I20240502 07:58:14.382555 5081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.68:39521 every 8 connection(s)
I20240502 07:58:14.383329 4954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240502 07:58:14.390949 5082 heartbeater.cc:347] Connected to a master server at 127.3.139.126:39833
I20240502 07:58:14.391149 5082 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:14.391463 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 4954
I20240502 07:58:14.391554 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240502 07:58:14.391646 5082 heartbeater.cc:511] Master 127.3.139.126:39833 requested a full tablet report, sending...
I20240502 07:58:14.392568 4501 ts_manager.cc:194] Registered new tserver with Master: bca5cc35c80a4c0b9cf5f476b9c3b741 (127.3.139.68:39521)
I20240502 07:58:14.393117 4501 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.68:52931
I20240502 07:58:14.393568 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.69:0
--local_ip_for_outbound_sockets=127.3.139.69
--webserver_interface=127.3.139.69
--webserver_port=0
--tserver_master_addrs=127.3.139.126:39833
--builtin_ntp_servers=127.3.139.84:42539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240502 07:58:14.490798 5085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:14.490975 5085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:14.491021 5085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:14.493938 5085 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:58:14.493994 5085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:14.494071 5085 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.69
I20240502 07:58:14.497427 5085 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:42539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.69:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.3.139.69
--webserver_port=0
--tserver_master_addrs=127.3.139.126:39833
--never_fsync=true
--heap_profile_path=/tmp/kudu.5085
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.69
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:14.498104 5085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:14.499035 5085 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:14.506173 5090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:14.506562 5091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:14.506956 5093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:14.507431 5085 server_base.cc:1034] running on GCE node
I20240502 07:58:14.507747 5085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:14.508256 5085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:14.509419 5085 hybrid_clock.cc:648] HybridClock initialized: now 1714636694509403 us; error 28 us; skew 500 ppm
I20240502 07:58:14.510986 5085 webserver.cc:460] Webserver started at http://127.3.139.69:37767/ using document root <none> and password file <none>
I20240502 07:58:14.511457 5085 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:14.511512 5085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:14.511656 5085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:14.513044 5085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "17e7d8627cd14b459396c628a3182958"
format_stamp: "Formatted at 2024-05-02 07:58:14 on dist-test-slave-twcr"
I20240502 07:58:14.513442 5085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "17e7d8627cd14b459396c628a3182958"
format_stamp: "Formatted at 2024-05-02 07:58:14 on dist-test-slave-twcr"
I20240502 07:58:14.516296 5085 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20240502 07:58:14.518118 5099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:14.518965 5085 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20240502 07:58:14.519074 5085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "17e7d8627cd14b459396c628a3182958"
format_stamp: "Formatted at 2024-05-02 07:58:14 on dist-test-slave-twcr"
I20240502 07:58:14.519162 5085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:14.537556 5085 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:14.538007 5085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:14.538120 5085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:14.538529 5085 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:14.539327 5085 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:14.539371 5085 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:14.539404 5085 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:14.539424 5085 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:14.546784 5085 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.69:37589
I20240502 07:58:14.546835 5212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.69:37589 every 8 connection(s)
I20240502 07:58:14.547628 5085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240502 07:58:14.547822 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 5085
I20240502 07:58:14.547915 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20240502 07:58:14.555253 5213 heartbeater.cc:347] Connected to a master server at 127.3.139.126:39833
I20240502 07:58:14.555444 5213 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:14.555876 5213 heartbeater.cc:511] Master 127.3.139.126:39833 requested a full tablet report, sending...
I20240502 07:58:14.556713 4501 ts_manager.cc:194] Registered new tserver with Master: 17e7d8627cd14b459396c628a3182958 (127.3.139.69:37589)
I20240502 07:58:14.557245 4501 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.69:34815
I20240502 07:58:14.561532 3629 external_mini_cluster.cc:930] 5 TS(s) registered with all masters
I20240502 07:58:14.569232 4501 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38750:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20240502 07:58:14.584673 4885 tablet_service.cc:1467] Processing CreateTablet for tablet b843f2436b6d4551811af07f34514f22 (DEFAULT_TABLE table=TestTable [id=2c6121aa3a834ff387ec8b0d6d611703]), partition=RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:14.585592 4885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b843f2436b6d4551811af07f34514f22. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:14.585335 4754 tablet_service.cc:1467] Processing CreateTablet for tablet b843f2436b6d4551811af07f34514f22 (DEFAULT_TABLE table=TestTable [id=2c6121aa3a834ff387ec8b0d6d611703]), partition=RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:14.586125 4754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b843f2436b6d4551811af07f34514f22. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:14.586549 4623 tablet_service.cc:1467] Processing CreateTablet for tablet b843f2436b6d4551811af07f34514f22 (DEFAULT_TABLE table=TestTable [id=2c6121aa3a834ff387ec8b0d6d611703]), partition=RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:14.587221 4623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b843f2436b6d4551811af07f34514f22. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:14.591665 5230 tablet_bootstrap.cc:492] T b843f2436b6d4551811af07f34514f22 P bf6bb6c77cac4487bb71e7b2ea1b1690: Bootstrap starting.
I20240502 07:58:14.591778 5231 tablet_bootstrap.cc:492] T b843f2436b6d4551811af07f34514f22 P 73cf6b5f71d64fb29b2ee84379fe4e04: Bootstrap starting.
I20240502 07:58:14.592139 5232 tablet_bootstrap.cc:492] T b843f2436b6d4551811af07f34514f22 P 3b0cf2effa6144788c6182357264817b: Bootstrap starting.
I20240502 07:58:14.593855 5232 tablet_bootstrap.cc:654] T b843f2436b6d4551811af07f34514f22 P 3b0cf2effa6144788c6182357264817b: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:14.593878 5230 tablet_bootstrap.cc:654] T b843f2436b6d4551811af07f34514f22 P bf6bb6c77cac4487bb71e7b2ea1b1690: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:14.594617 5232 log.cc:824] T b843f2436b6d4551811af07f34514f22 P 3b0cf2effa6144788c6182357264817b: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:14.594617 5230 log.cc:824] T b843f2436b6d4551811af07f34514f22 P bf6bb6c77cac4487bb71e7b2ea1b1690: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:14.596246 5232 tablet_bootstrap.cc:492] T b843f2436b6d4551811af07f34514f22 P 3b0cf2effa6144788c6182357264817b: No bootstrap required, opened a new log
I20240502 07:58:14.596410 5232 ts_tablet_manager.cc:1367] T b843f2436b6d4551811af07f34514f22 P 3b0cf2effa6144788c6182357264817b: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240502 07:58:14.596792 5231 tablet_bootstrap.cc:654] T b843f2436b6d4551811af07f34514f22 P 73cf6b5f71d64fb29b2ee84379fe4e04: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:14.597494 5231 log.cc:824] T b843f2436b6d4551811af07f34514f22 P 73cf6b5f71d64fb29b2ee84379fe4e04: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:14.598994 5231 tablet_bootstrap.cc:492] T b843f2436b6d4551811af07f34514f22 P 73cf6b5f71d64fb29b2ee84379fe4e04: No bootstrap required, opened a new log
I20240502 07:58:14.599193 5231 ts_tablet_manager.cc:1367] T b843f2436b6d4551811af07f34514f22 P 73cf6b5f71d64fb29b2ee84379fe4e04: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.001s
I20240502 07:58:14.599846 5232 raft_consensus.cc:348] T b843f2436b6d4551811af07f34514f22 P 3b0cf2effa6144788c6182357264817b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf6bb6c77cac4487bb71e7b2ea1b1690" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 41209 } } peers { permanent_uuid: "73cf6b5f71d64fb29b2ee84379fe4e04" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46151 } } peers { permanent_uuid: "3b0cf2effa6144788c6182357264817b" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35895 } }
I20240502 07:58:14.600018 5230 tablet_bootstrap.cc:492] T b843f2436b6d4551811af07f34514f22 P bf6bb6c77cac4487bb71e7b2ea1b1690: No bootstrap required, opened a new log
I20240502 07:58:14.600068 5232 raft_consensus.cc:374] T b843f2436b6d4551811af07f34514f22 P 3b0cf2effa6144788c6182357264817b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:14.600126 5232 raft_consensus.cc:729] T b843f2436b6d4551811af07f34514f22 P 3b0cf2effa6144788c6182357264817b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b0cf2effa6144788c6182357264817b, State: Initialized, Role: FOLLOWER
I20240502 07:58:14.600159 5230 ts_tablet_manager.cc:1367] T b843f2436b6d4551811af07f34514f22 P bf6bb6c77cac4487bb71e7b2ea1b1690: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.000s
I20240502 07:58:14.600519 5232 consensus_queue.cc:260] T b843f2436b6d4551811af07f34514f22 P 3b0cf2effa6144788c6182357264817b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf6bb6c77cac4487bb71e7b2ea1b1690" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 41209 } } peers { permanent_uuid: "73cf6b5f71d64fb29b2ee84379fe4e04" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46151 } } peers { permanent_uuid: "3b0cf2effa6144788c6182357264817b" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35895 } }
I20240502 07:58:14.601140 4689 heartbeater.cc:503] Master 127.3.139.126:39833 was elected leader, sending a full tablet report...
I20240502 07:58:14.601351 5232 ts_tablet_manager.cc:1398] T b843f2436b6d4551811af07f34514f22 P 3b0cf2effa6144788c6182357264817b: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20240502 07:58:14.602082 5231 raft_consensus.cc:348] T b843f2436b6d4551811af07f34514f22 P 73cf6b5f71d64fb29b2ee84379fe4e04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf6bb6c77cac4487bb71e7b2ea1b1690" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 41209 } } peers { permanent_uuid: "73cf6b5f71d64fb29b2ee84379fe4e04" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46151 } } peers { permanent_uuid: "3b0cf2effa6144788c6182357264817b" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35895 } }
I20240502 07:58:14.602257 5231 raft_consensus.cc:374] T b843f2436b6d4551811af07f34514f22 P 73cf6b5f71d64fb29b2ee84379fe4e04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:14.602314 5231 raft_consensus.cc:729] T b843f2436b6d4551811af07f34514f22 P 73cf6b5f71d64fb29b2ee84379fe4e04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73cf6b5f71d64fb29b2ee84379fe4e04, State: Initialized, Role: FOLLOWER
I20240502 07:58:14.602650 5231 consensus_queue.cc:260] T b843f2436b6d4551811af07f34514f22 P 73cf6b5f71d64fb29b2ee84379fe4e04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf6bb6c77cac4487bb71e7b2ea1b1690" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 41209 } } peers { permanent_uuid: "73cf6b5f71d64fb29b2ee84379fe4e04" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46151 } } peers { permanent_uuid: "3b0cf2effa6144788c6182357264817b" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35895 } }
I20240502 07:58:14.603435 5231 ts_tablet_manager.cc:1398] T b843f2436b6d4551811af07f34514f22 P 73cf6b5f71d64fb29b2ee84379fe4e04: Time spent starting tablet: real 0.004s user 0.001s sys 0.003s
I20240502 07:58:14.603554 4820 heartbeater.cc:503] Master 127.3.139.126:39833 was elected leader, sending a full tablet report...
I20240502 07:58:14.603515 5230 raft_consensus.cc:348] T b843f2436b6d4551811af07f34514f22 P bf6bb6c77cac4487bb71e7b2ea1b1690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf6bb6c77cac4487bb71e7b2ea1b1690" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 41209 } } peers { permanent_uuid: "73cf6b5f71d64fb29b2ee84379fe4e04" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46151 } } peers { permanent_uuid: "3b0cf2effa6144788c6182357264817b" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35895 } }
I20240502 07:58:14.603682 5230 raft_consensus.cc:374] T b843f2436b6d4551811af07f34514f22 P bf6bb6c77cac4487bb71e7b2ea1b1690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:14.603727 5230 raft_consensus.cc:729] T b843f2436b6d4551811af07f34514f22 P bf6bb6c77cac4487bb71e7b2ea1b1690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf6bb6c77cac4487bb71e7b2ea1b1690, State: Initialized, Role: FOLLOWER
I20240502 07:58:14.604084 5230 consensus_queue.cc:260] T b843f2436b6d4551811af07f34514f22 P bf6bb6c77cac4487bb71e7b2ea1b1690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf6bb6c77cac4487bb71e7b2ea1b1690" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 41209 } } peers { permanent_uuid: "73cf6b5f71d64fb29b2ee84379fe4e04" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46151 } } peers { permanent_uuid: "3b0cf2effa6144788c6182357264817b" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35895 } }
I20240502 07:58:14.604749 4951 heartbeater.cc:503] Master 127.3.139.126:39833 was elected leader, sending a full tablet report...
I20240502 07:58:14.604853 5230 ts_tablet_manager.cc:1398] T b843f2436b6d4551811af07f34514f22 P bf6bb6c77cac4487bb71e7b2ea1b1690: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20240502 07:58:14.650478 4690 tablet.cc:2369] T b843f2436b6d4551811af07f34514f22 P 3b0cf2effa6144788c6182357264817b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240502 07:58:14.728169 4952 tablet.cc:2369] T b843f2436b6d4551811af07f34514f22 P bf6bb6c77cac4487bb71e7b2ea1b1690: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240502 07:58:14.827528 4821 tablet.cc:2369] T b843f2436b6d4551811af07f34514f22 P 73cf6b5f71d64fb29b2ee84379fe4e04: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 07:58:15.008183 5238 raft_consensus.cc:482] T b843f2436b6d4551811af07f34514f22 P bf6bb6c77cac4487bb71e7b2ea1b1690 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:15.008349 5238 raft_consensus.cc:504] T b843f2436b6d4551811af07f34514f22 P bf6bb6c77cac4487bb71e7b2ea1b1690 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf6bb6c77cac4487bb71e7b2ea1b1690" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 41209 } } peers { permanent_uuid: "73cf6b5f71d64fb29b2ee84379fe4e04" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46151 } } peers { permanent_uuid: "3b0cf2effa6144788c6182357264817b" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35895 } }
I20240502 07:58:15.009428 5238 leader_election.cc:288] T b843f2436b6d4551811af07f34514f22 P bf6bb6c77cac4487bb71e7b2ea1b1690 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73cf6b5f71d64fb29b2ee84379fe4e04 (127.3.139.66:46151), 3b0cf2effa6144788c6182357264817b (127.3.139.65:35895)
I20240502 07:58:15.012945 4774 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b843f2436b6d4551811af07f34514f22" candidate_uuid: "bf6bb6c77cac4487bb71e7b2ea1b1690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73cf6b5f71d64fb29b2ee84379fe4e04" is_pre_election: true
I20240502 07:58:15.013168 4643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b843f2436b6d4551811af07f34514f22" candidate_uuid: "bf6bb6c77cac4487bb71e7b2ea1b1690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b0cf2effa6144788c6182357264817b" is_pre_election: true
I20240502 07:58:15.013227 4774 raft_consensus.cc:2454] T b843f2436b6d4551811af07f34514f22 P 73cf6b5f71d64fb29b2ee84379fe4e04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf6bb6c77cac4487bb71e7b2ea1b1690 in term 0.
I20240502 07:58:15.013413 4643 raft_consensus.cc:2454] T b843f2436b6d4551811af07f34514f22 P 3b0cf2effa6144788c6182357264817b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf6bb6c77cac4487bb71e7b2ea1b1690 in term 0.
I20240502 07:58:15.013655 4838 leader_election.cc:302] T b843f2436b6d4551811af07f34514f22 P bf6bb6c77cac4487bb71e7b2ea1b1690 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73cf6b5f71d64fb29b2ee84379fe4e04, bf6bb6c77cac4487bb71e7b2ea1b1690; no voters:
I20240502 07:58:15.013911 5238 raft_consensus.cc:2781] T b843f2436b6d4551811af07f34514f22 P bf6bb6c77cac4487bb71e7b2ea1b1690 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:15.013979 5238 raft_consensus.cc:482] T b843f2436b6d4551811af07f34514f22 P bf6bb6c77cac4487bb71e7b2ea1b1690 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:15.014029 5238 raft_consensus.cc:3037] T b843f2436b6d4551811af07f34514f22 P bf6bb6c77cac4487bb71e7b2ea1b1690 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:15.014917 5238 raft_consensus.cc:504] T b843f2436b6d4551811af07f34514f22 P bf6bb6c77cac4487bb71e7b2ea1b1690 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf6bb6c77cac4487bb71e7b2ea1b1690" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 41209 } } peers { permanent_uuid: "73cf6b5f71d64fb29b2ee84379fe4e04" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46151 } } peers { permanent_uuid: "3b0cf2effa6144788c6182357264817b" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35895 } }
I20240502 07:58:15.015237 5238 leader_election.cc:288] T b843f2436b6d4551811af07f34514f22 P bf6bb6c77cac4487bb71e7b2ea1b1690 [CANDIDATE]: Term 1 election: Requested vote from peers 73cf6b5f71d64fb29b2ee84379fe4e04 (127.3.139.66:46151), 3b0cf2effa6144788c6182357264817b (127.3.139.65:35895)
I20240502 07:58:15.015527 4774 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b843f2436b6d4551811af07f34514f22" candidate_uuid: "bf6bb6c77cac4487bb71e7b2ea1b1690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73cf6b5f71d64fb29b2ee84379fe4e04"
I20240502 07:58:15.015585 4643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b843f2436b6d4551811af07f34514f22" candidate_uuid: "bf6bb6c77cac4487bb71e7b2ea1b1690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b0cf2effa6144788c6182357264817b"
I20240502 07:58:15.015633 4774 raft_consensus.cc:3037] T b843f2436b6d4551811af07f34514f22 P 73cf6b5f71d64fb29b2ee84379fe4e04 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:15.015694 4643 raft_consensus.cc:3037] T b843f2436b6d4551811af07f34514f22 P 3b0cf2effa6144788c6182357264817b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:15.016413 4774 raft_consensus.cc:2454] T b843f2436b6d4551811af07f34514f22 P 73cf6b5f71d64fb29b2ee84379fe4e04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf6bb6c77cac4487bb71e7b2ea1b1690 in term 1.
I20240502 07:58:15.016458 4643 raft_consensus.cc:2454] T b843f2436b6d4551811af07f34514f22 P 3b0cf2effa6144788c6182357264817b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf6bb6c77cac4487bb71e7b2ea1b1690 in term 1.
I20240502 07:58:15.016652 4838 leader_election.cc:302] T b843f2436b6d4551811af07f34514f22 P bf6bb6c77cac4487bb71e7b2ea1b1690 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73cf6b5f71d64fb29b2ee84379fe4e04, bf6bb6c77cac4487bb71e7b2ea1b1690; no voters:
I20240502 07:58:15.016789 5238 raft_consensus.cc:2781] T b843f2436b6d4551811af07f34514f22 P bf6bb6c77cac4487bb71e7b2ea1b1690 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:15.016976 5238 raft_consensus.cc:686] T b843f2436b6d4551811af07f34514f22 P bf6bb6c77cac4487bb71e7b2ea1b1690 [term 1 LEADER]: Becoming Leader. State: Replica: bf6bb6c77cac4487bb71e7b2ea1b1690, State: Running, Role: LEADER
I20240502 07:58:15.017247 5238 consensus_queue.cc:237] T b843f2436b6d4551811af07f34514f22 P bf6bb6c77cac4487bb71e7b2ea1b1690 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf6bb6c77cac4487bb71e7b2ea1b1690" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 41209 } } peers { permanent_uuid: "73cf6b5f71d64fb29b2ee84379fe4e04" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46151 } } peers { permanent_uuid: "3b0cf2effa6144788c6182357264817b" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35895 } }
I20240502 07:58:15.019749 4499 catalog_manager.cc:5484] T b843f2436b6d4551811af07f34514f22 P bf6bb6c77cac4487bb71e7b2ea1b1690 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf6bb6c77cac4487bb71e7b2ea1b1690 (127.3.139.67). New cstate: current_term: 1 leader_uuid: "bf6bb6c77cac4487bb71e7b2ea1b1690" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf6bb6c77cac4487bb71e7b2ea1b1690" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 41209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73cf6b5f71d64fb29b2ee84379fe4e04" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b0cf2effa6144788c6182357264817b" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35895 } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:15.134265 3629 external_mini_cluster.cc:930] 5 TS(s) registered with all masters
I20240502 07:58:15.135736 3629 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 3b0cf2effa6144788c6182357264817b to finish bootstrapping
I20240502 07:58:15.140703 3629 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 73cf6b5f71d64fb29b2ee84379fe4e04 to finish bootstrapping
I20240502 07:58:15.144618 3629 ts_itest-base.cc:233] Waiting for 1 tablets on tserver bf6bb6c77cac4487bb71e7b2ea1b1690 to finish bootstrapping
I20240502 07:58:15.148555 3629 ts_itest-base.cc:233] Waiting for 0 tablets on tserver bca5cc35c80a4c0b9cf5f476b9c3b741 to finish bootstrapping
I20240502 07:58:15.153533 3629 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 17e7d8627cd14b459396c628a3182958 to finish bootstrapping
I20240502 07:58:15.319763 4623 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:15.320219 5016 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:15.320593 4754 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:15.320780 5147 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:15.335126 4885 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:15.394755 5082 heartbeater.cc:503] Master 127.3.139.126:39833 was elected leader, sending a full tablet report...
I20240502 07:58:15.404923 5016 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:15.405418 4885 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:15.405735 5147 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:15.405738 4623 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:15.411975 4754 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:15.428963 5238 consensus_queue.cc:1035] T b843f2436b6d4551811af07f34514f22 P bf6bb6c77cac4487bb71e7b2ea1b1690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73cf6b5f71d64fb29b2ee84379fe4e04" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:15.434221 5238 consensus_queue.cc:1035] T b843f2436b6d4551811af07f34514f22 P bf6bb6c77cac4487bb71e7b2ea1b1690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b0cf2effa6144788c6182357264817b" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 35895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:15.500981 5147 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:15.501178 4885 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:15.502229 5016 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:15.504495 4754 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:15.511909 4623 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:15.558821 5213 heartbeater.cc:503] Master 127.3.139.126:39833 was elected leader, sending a full tablet report...
I20240502 07:58:15.702759 5147 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:15.703001 5016 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:15.704383 4885 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:15.706187 4623 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:15.706276 4754 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:15.788008 5016 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:15.788450 4754 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:15.789337 5147 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:15.792017 4623 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:15.796212 4885 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:15.900234 5147 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:15.906984 4623 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:15.907637 4885 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:15.909617 4754 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:15.911394 5016 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:15.942106 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 4561
I20240502 07:58:15.946732 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 4692
I20240502 07:58:15.951454 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 4823
I20240502 07:58:15.955673 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 4954
I20240502 07:58:15.959744 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 5085
I20240502 07:58:15.963986 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 4470
2024-05-02T07:58:15Z chronyd exiting
[ OK ] RebalanceStartSafetyTest.TooManyIgnoredTservers/1 (2463 ms)
[----------] 1 test from RebalanceStartSafetyTest (2463 ms total)
[----------] 1 test from RebalanceParamTest
[ RUN ] RebalanceParamTest.Rebalance/7
I20240502 07:58:15.983613 3629 test_util.cc:268] Using random seed: -147925509
I20240502 07:58:15.984273 3629 ts_itest-base.cc:112] Starting cluster with:
I20240502 07:58:15.984310 3629 ts_itest-base.cc:113] --------------
I20240502 07:58:15.984324 3629 ts_itest-base.cc:114] 7 tablet servers
I20240502 07:58:15.984337 3629 ts_itest-base.cc:115] 5 replicas per TS
I20240502 07:58:15.984349 3629 ts_itest-base.cc:116] --------------
2024-05-02T07:58:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-02T07:58:15Z Disabled control of system clock
I20240502 07:58:15.990880 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.139.126:34225
--webserver_interface=127.3.139.126
--webserver_port=0
--builtin_ntp_servers=127.3.139.84:41319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.139.126:34225
--allow_unsafe_replication_factor
--enable_range_replica_placement=false
--raft_prepare_replacement_before_eviction=true
--tserver_unresponsive_timeout_ms=3000 with env {}
W20240502 07:58:16.092068 5457 flags.cc:425] Enabled unsafe flag: --allow_unsafe_replication_factor=true
W20240502 07:58:16.092262 5457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:16.092296 5457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:16.092351 5457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:16.095237 5457 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:58:16.095300 5457 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240502 07:58:16.095331 5457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:16.095364 5457 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240502 07:58:16.095396 5457 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240502 07:58:16.098793 5457 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:41319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal
--allow_unsafe_replication_factor=true
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.3.139.126:34225
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.139.126:34225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.3.139.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.5457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:16.099510 5457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:16.100458 5457 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:16.106637 5465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:16.106982 5462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:16.107298 5463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:16.107772 5457 server_base.cc:1034] running on GCE node
I20240502 07:58:16.108158 5457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:16.108881 5457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:16.110021 5457 hybrid_clock.cc:648] HybridClock initialized: now 1714636696110009 us; error 32 us; skew 500 ppm
I20240502 07:58:16.111611 5457 webserver.cc:460] Webserver started at http://127.3.139.126:33745/ using document root <none> and password file <none>
I20240502 07:58:16.112071 5457 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:16.112123 5457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:16.112272 5457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:16.113644 5457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "c5d1643b3f5747ccb4514a400dcc82b4"
format_stamp: "Formatted at 2024-05-02 07:58:16 on dist-test-slave-twcr"
I20240502 07:58:16.114058 5457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "c5d1643b3f5747ccb4514a400dcc82b4"
format_stamp: "Formatted at 2024-05-02 07:58:16 on dist-test-slave-twcr"
I20240502 07:58:16.116931 5457 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 07:58:16.118752 5471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:16.119632 5457 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:16.119735 5457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal
uuid: "c5d1643b3f5747ccb4514a400dcc82b4"
format_stamp: "Formatted at 2024-05-02 07:58:16 on dist-test-slave-twcr"
I20240502 07:58:16.119801 5457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:16.133575 5457 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:16.134043 5457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:16.134164 5457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:16.139678 5457 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.126:34225
I20240502 07:58:16.139693 5523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.126:34225 every 8 connection(s)
I20240502 07:58:16.140426 5457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240502 07:58:16.142669 5524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:16.144754 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 5457
I20240502 07:58:16.144870 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240502 07:58:16.147367 5524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5d1643b3f5747ccb4514a400dcc82b4: Bootstrap starting.
I20240502 07:58:16.149116 5524 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5d1643b3f5747ccb4514a400dcc82b4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:16.149706 5524 log.cc:824] T 00000000000000000000000000000000 P c5d1643b3f5747ccb4514a400dcc82b4: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:16.151389 5524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5d1643b3f5747ccb4514a400dcc82b4: No bootstrap required, opened a new log
I20240502 07:58:16.153826 5524 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5d1643b3f5747ccb4514a400dcc82b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5d1643b3f5747ccb4514a400dcc82b4" member_type: VOTER last_known_addr { host: "127.3.139.126" port: 34225 } }
I20240502 07:58:16.153982 5524 raft_consensus.cc:374] T 00000000000000000000000000000000 P c5d1643b3f5747ccb4514a400dcc82b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:16.154013 5524 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5d1643b3f5747ccb4514a400dcc82b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5d1643b3f5747ccb4514a400dcc82b4, State: Initialized, Role: FOLLOWER
I20240502 07:58:16.154314 5524 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5d1643b3f5747ccb4514a400dcc82b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5d1643b3f5747ccb4514a400dcc82b4" member_type: VOTER last_known_addr { host: "127.3.139.126" port: 34225 } }
I20240502 07:58:16.154407 5524 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5d1643b3f5747ccb4514a400dcc82b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 07:58:16.154445 5524 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5d1643b3f5747ccb4514a400dcc82b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 07:58:16.154510 5524 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5d1643b3f5747ccb4514a400dcc82b4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:16.155102 5524 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5d1643b3f5747ccb4514a400dcc82b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5d1643b3f5747ccb4514a400dcc82b4" member_type: VOTER last_known_addr { host: "127.3.139.126" port: 34225 } }
I20240502 07:58:16.155380 5524 leader_election.cc:302] T 00000000000000000000000000000000 P c5d1643b3f5747ccb4514a400dcc82b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5d1643b3f5747ccb4514a400dcc82b4; no voters:
I20240502 07:58:16.155601 5524 leader_election.cc:288] T 00000000000000000000000000000000 P c5d1643b3f5747ccb4514a400dcc82b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 07:58:16.155684 5529 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5d1643b3f5747ccb4514a400dcc82b4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:16.156175 5529 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5d1643b3f5747ccb4514a400dcc82b4 [term 1 LEADER]: Becoming Leader. State: Replica: c5d1643b3f5747ccb4514a400dcc82b4, State: Running, Role: LEADER
I20240502 07:58:16.156491 5524 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5d1643b3f5747ccb4514a400dcc82b4 [sys.catalog]: configured and running, proceeding with master startup.
I20240502 07:58:16.156659 5529 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5d1643b3f5747ccb4514a400dcc82b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5d1643b3f5747ccb4514a400dcc82b4" member_type: VOTER last_known_addr { host: "127.3.139.126" port: 34225 } }
I20240502 07:58:16.158131 5531 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5d1643b3f5747ccb4514a400dcc82b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5d1643b3f5747ccb4514a400dcc82b4. Latest consensus state: current_term: 1 leader_uuid: "c5d1643b3f5747ccb4514a400dcc82b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5d1643b3f5747ccb4514a400dcc82b4" member_type: VOTER last_known_addr { host: "127.3.139.126" port: 34225 } } }
I20240502 07:58:16.158237 5531 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5d1643b3f5747ccb4514a400dcc82b4 [sys.catalog]: This master's current role is: LEADER
I20240502 07:58:16.158383 5530 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5d1643b3f5747ccb4514a400dcc82b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c5d1643b3f5747ccb4514a400dcc82b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5d1643b3f5747ccb4514a400dcc82b4" member_type: VOTER last_known_addr { host: "127.3.139.126" port: 34225 } } }
I20240502 07:58:16.158617 5530 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5d1643b3f5747ccb4514a400dcc82b4 [sys.catalog]: This master's current role is: LEADER
W20240502 07:58:16.160408 5543 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c5d1643b3f5747ccb4514a400dcc82b4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240502 07:58:16.160466 5543 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20240502 07:58:16.160542 5544 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240502 07:58:16.161437 5544 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240502 07:58:16.165540 5544 catalog_manager.cc:1347] Generated new cluster ID: ae96605da4a04bb284e68bc00d0706c2
I20240502 07:58:16.165601 5544 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240502 07:58:16.181797 5544 catalog_manager.cc:1370] Generated new certificate authority record
I20240502 07:58:16.182530 5544 catalog_manager.cc:1504] Loading token signing keys...
I20240502 07:58:16.190084 5544 catalog_manager.cc:5857] T 00000000000000000000000000000000 P c5d1643b3f5747ccb4514a400dcc82b4: Generated new TSK 0
I20240502 07:58:16.190589 5544 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240502 07:58:16.202040 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.65:0
--local_ip_for_outbound_sockets=127.3.139.65
--webserver_interface=127.3.139.65
--webserver_port=0
--tserver_master_addrs=127.3.139.126:34225
--builtin_ntp_servers=127.3.139.84:41319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240502 07:58:16.298256 5548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:16.298432 5548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:16.298487 5548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:16.301378 5548 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:58:16.301436 5548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:16.301517 5548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.65
I20240502 07:58:16.305202 5548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:41319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.3.139.65
--webserver_port=0
--tserver_master_addrs=127.3.139.126:34225
--never_fsync=true
--heap_profile_path=/tmp/kudu.5548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.65
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:16.305943 5548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:16.306923 5548 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:16.314693 5553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:16.314960 5554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:16.315392 5556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:16.315774 5548 server_base.cc:1034] running on GCE node
I20240502 07:58:16.316078 5548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:16.316531 5548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:16.317682 5548 hybrid_clock.cc:648] HybridClock initialized: now 1714636696317671 us; error 23 us; skew 500 ppm
I20240502 07:58:16.319406 5548 webserver.cc:460] Webserver started at http://127.3.139.65:33059/ using document root <none> and password file <none>
I20240502 07:58:16.319945 5548 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:16.319999 5548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:16.320226 5548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:16.321734 5548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "91e82003a656442189a329b29846a841"
format_stamp: "Formatted at 2024-05-02 07:58:16 on dist-test-slave-twcr"
I20240502 07:58:16.322177 5548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "91e82003a656442189a329b29846a841"
format_stamp: "Formatted at 2024-05-02 07:58:16 on dist-test-slave-twcr"
I20240502 07:58:16.325134 5548 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20240502 07:58:16.327126 5562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:16.328107 5548 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20240502 07:58:16.328214 5548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "91e82003a656442189a329b29846a841"
format_stamp: "Formatted at 2024-05-02 07:58:16 on dist-test-slave-twcr"
I20240502 07:58:16.328286 5548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:16.342602 5548 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:16.343178 5548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:16.343319 5548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:16.343772 5548 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:16.344552 5548 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:16.344596 5548 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:16.344645 5548 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:16.344673 5548 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:16.352485 5548 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.65:43429
I20240502 07:58:16.352515 5675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.65:43429 every 8 connection(s)
I20240502 07:58:16.353272 5548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240502 07:58:16.356760 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 5548
I20240502 07:58:16.356879 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240502 07:58:16.358866 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.66:0
--local_ip_for_outbound_sockets=127.3.139.66
--webserver_interface=127.3.139.66
--webserver_port=0
--tserver_master_addrs=127.3.139.126:34225
--builtin_ntp_servers=127.3.139.84:41319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 07:58:16.361867 5676 heartbeater.cc:347] Connected to a master server at 127.3.139.126:34225
I20240502 07:58:16.362061 5676 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:16.362498 5676 heartbeater.cc:511] Master 127.3.139.126:34225 requested a full tablet report, sending...
I20240502 07:58:16.363981 5488 ts_manager.cc:194] Registered new tserver with Master: 91e82003a656442189a329b29846a841 (127.3.139.65:43429)
I20240502 07:58:16.365542 5488 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.65:58505
W20240502 07:58:16.456337 5679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:16.456506 5679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:16.456547 5679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:16.459607 5679 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:58:16.459661 5679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:16.459721 5679 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.66
I20240502 07:58:16.463126 5679 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:41319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.3.139.66
--webserver_port=0
--tserver_master_addrs=127.3.139.126:34225
--never_fsync=true
--heap_profile_path=/tmp/kudu.5679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.66
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:16.463773 5679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:16.464684 5679 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:16.471707 5685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:16.472059 5687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:16.472440 5684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:16.472815 5679 server_base.cc:1034] running on GCE node
I20240502 07:58:16.473115 5679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:16.473588 5679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:16.474730 5679 hybrid_clock.cc:648] HybridClock initialized: now 1714636696474717 us; error 27 us; skew 500 ppm
I20240502 07:58:16.476419 5679 webserver.cc:460] Webserver started at http://127.3.139.66:38165/ using document root <none> and password file <none>
I20240502 07:58:16.476883 5679 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:16.476935 5679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:16.477102 5679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:16.478525 5679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "eda55a8566c447cfa749457825b70688"
format_stamp: "Formatted at 2024-05-02 07:58:16 on dist-test-slave-twcr"
I20240502 07:58:16.478955 5679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "eda55a8566c447cfa749457825b70688"
format_stamp: "Formatted at 2024-05-02 07:58:16 on dist-test-slave-twcr"
I20240502 07:58:16.481863 5679 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20240502 07:58:16.483809 5693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:16.484723 5679 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20240502 07:58:16.484822 5679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "eda55a8566c447cfa749457825b70688"
format_stamp: "Formatted at 2024-05-02 07:58:16 on dist-test-slave-twcr"
I20240502 07:58:16.484905 5679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:16.505719 5679 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:16.506218 5679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:16.506345 5679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:16.506776 5679 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:16.507579 5679 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:16.507623 5679 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:16.507659 5679 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:16.507685 5679 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:16.515740 5679 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.66:39925
I20240502 07:58:16.515777 5806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.66:39925 every 8 connection(s)
I20240502 07:58:16.516520 5679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240502 07:58:16.523854 5807 heartbeater.cc:347] Connected to a master server at 127.3.139.126:34225
I20240502 07:58:16.523933 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 5679
I20240502 07:58:16.524040 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240502 07:58:16.524044 5807 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:16.524478 5807 heartbeater.cc:511] Master 127.3.139.126:34225 requested a full tablet report, sending...
I20240502 07:58:16.525332 5488 ts_manager.cc:194] Registered new tserver with Master: eda55a8566c447cfa749457825b70688 (127.3.139.66:39925)
I20240502 07:58:16.525956 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.67:0
--local_ip_for_outbound_sockets=127.3.139.67
--webserver_interface=127.3.139.67
--webserver_port=0
--tserver_master_addrs=127.3.139.126:34225
--builtin_ntp_servers=127.3.139.84:41319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 07:58:16.526013 5488 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.66:55443
W20240502 07:58:16.621215 5810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:16.621378 5810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:16.621438 5810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:16.624397 5810 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:58:16.624454 5810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:16.624534 5810 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.67
I20240502 07:58:16.627887 5810 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:41319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.3.139.67
--webserver_port=0
--tserver_master_addrs=127.3.139.126:34225
--never_fsync=true
--heap_profile_path=/tmp/kudu.5810
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.67
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:16.628556 5810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:16.629453 5810 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:16.636351 5815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:16.636673 5816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:16.636993 5818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:16.637436 5810 server_base.cc:1034] running on GCE node
I20240502 07:58:16.637729 5810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:16.638159 5810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:16.639302 5810 hybrid_clock.cc:648] HybridClock initialized: now 1714636696639292 us; error 19 us; skew 500 ppm
I20240502 07:58:16.640817 5810 webserver.cc:460] Webserver started at http://127.3.139.67:34305/ using document root <none> and password file <none>
I20240502 07:58:16.641290 5810 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:16.641345 5810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:16.641492 5810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:16.642848 5810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "89bf5c0a6b2a4d949851089446f17b96"
format_stamp: "Formatted at 2024-05-02 07:58:16 on dist-test-slave-twcr"
I20240502 07:58:16.643278 5810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "89bf5c0a6b2a4d949851089446f17b96"
format_stamp: "Formatted at 2024-05-02 07:58:16 on dist-test-slave-twcr"
I20240502 07:58:16.646010 5810 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20240502 07:58:16.647823 5824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:16.648620 5810 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:16.648712 5810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "89bf5c0a6b2a4d949851089446f17b96"
format_stamp: "Formatted at 2024-05-02 07:58:16 on dist-test-slave-twcr"
I20240502 07:58:16.648783 5810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:16.673868 5810 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:16.674341 5810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:16.674470 5810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:16.674896 5810 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:16.675707 5810 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:16.675784 5810 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:16.675840 5810 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:16.675868 5810 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:16.683306 5810 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.67:37141
I20240502 07:58:16.683332 5937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.67:37141 every 8 connection(s)
I20240502 07:58:16.684054 5810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240502 07:58:16.690564 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 5810
I20240502 07:58:16.690665 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240502 07:58:16.691356 5938 heartbeater.cc:347] Connected to a master server at 127.3.139.126:34225
I20240502 07:58:16.691543 5938 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:16.692016 5938 heartbeater.cc:511] Master 127.3.139.126:34225 requested a full tablet report, sending...
I20240502 07:58:16.692535 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.68:0
--local_ip_for_outbound_sockets=127.3.139.68
--webserver_interface=127.3.139.68
--webserver_port=0
--tserver_master_addrs=127.3.139.126:34225
--builtin_ntp_servers=127.3.139.84:41319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 07:58:16.692843 5488 ts_manager.cc:194] Registered new tserver with Master: 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67:37141)
I20240502 07:58:16.693338 5488 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.67:42849
W20240502 07:58:16.789675 5941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:16.789835 5941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:16.789877 5941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:16.792742 5941 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:58:16.792796 5941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:16.792892 5941 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.68
I20240502 07:58:16.796267 5941 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:41319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.68:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.3.139.68
--webserver_port=0
--tserver_master_addrs=127.3.139.126:34225
--never_fsync=true
--heap_profile_path=/tmp/kudu.5941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.68
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:16.796939 5941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:16.797844 5941 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:16.804766 5949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:16.805094 5946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:16.805433 5947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:16.805868 5941 server_base.cc:1034] running on GCE node
I20240502 07:58:16.806178 5941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:16.806622 5941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:16.807754 5941 hybrid_clock.cc:648] HybridClock initialized: now 1714636696807738 us; error 33 us; skew 500 ppm
I20240502 07:58:16.809320 5941 webserver.cc:460] Webserver started at http://127.3.139.68:38531/ using document root <none> and password file <none>
I20240502 07:58:16.809785 5941 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:16.809839 5941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:16.809999 5941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:16.811453 5941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "d5bc7863cd294f77a932c4a40f98b79d"
format_stamp: "Formatted at 2024-05-02 07:58:16 on dist-test-slave-twcr"
I20240502 07:58:16.811856 5941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "d5bc7863cd294f77a932c4a40f98b79d"
format_stamp: "Formatted at 2024-05-02 07:58:16 on dist-test-slave-twcr"
I20240502 07:58:16.814620 5941 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240502 07:58:16.816465 5955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:16.817304 5941 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:16.817407 5941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "d5bc7863cd294f77a932c4a40f98b79d"
format_stamp: "Formatted at 2024-05-02 07:58:16 on dist-test-slave-twcr"
I20240502 07:58:16.817477 5941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:16.832620 5941 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:16.833091 5941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:16.833218 5941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:16.833649 5941 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:16.834390 5941 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:16.834434 5941 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:16.834479 5941 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:16.834507 5941 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:16.841799 5941 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.68:44287
I20240502 07:58:16.841818 6068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.68:44287 every 8 connection(s)
I20240502 07:58:16.842548 5941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240502 07:58:16.846889 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 5941
I20240502 07:58:16.846987 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240502 07:58:16.849074 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.69:0
--local_ip_for_outbound_sockets=127.3.139.69
--webserver_interface=127.3.139.69
--webserver_port=0
--tserver_master_addrs=127.3.139.126:34225
--builtin_ntp_servers=127.3.139.84:41319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 07:58:16.850255 6069 heartbeater.cc:347] Connected to a master server at 127.3.139.126:34225
I20240502 07:58:16.850471 6069 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:16.851034 6069 heartbeater.cc:511] Master 127.3.139.126:34225 requested a full tablet report, sending...
I20240502 07:58:16.852020 5488 ts_manager.cc:194] Registered new tserver with Master: d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68:44287)
I20240502 07:58:16.852604 5488 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.68:35919
W20240502 07:58:16.947849 6072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:16.948009 6072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:16.948055 6072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:16.950912 6072 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:58:16.950963 6072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:16.951071 6072 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.69
I20240502 07:58:16.954478 6072 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:41319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.69:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.3.139.69
--webserver_port=0
--tserver_master_addrs=127.3.139.126:34225
--never_fsync=true
--heap_profile_path=/tmp/kudu.6072
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.69
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:16.955163 6072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:16.956092 6072 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:16.963019 6077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:16.963336 6080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:16.963682 6078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:16.964113 6072 server_base.cc:1034] running on GCE node
I20240502 07:58:16.964418 6072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:16.964845 6072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:16.965991 6072 hybrid_clock.cc:648] HybridClock initialized: now 1714636696965968 us; error 34 us; skew 500 ppm
I20240502 07:58:16.967566 6072 webserver.cc:460] Webserver started at http://127.3.139.69:40479/ using document root <none> and password file <none>
I20240502 07:58:16.968022 6072 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:16.968071 6072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:16.968231 6072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:16.969619 6072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "e020e7ffe8ce423a8bdefec667161217"
format_stamp: "Formatted at 2024-05-02 07:58:16 on dist-test-slave-twcr"
I20240502 07:58:16.970031 6072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "e020e7ffe8ce423a8bdefec667161217"
format_stamp: "Formatted at 2024-05-02 07:58:16 on dist-test-slave-twcr"
I20240502 07:58:16.972831 6072 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 07:58:16.974668 6086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:16.975546 6072 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:16.975648 6072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "e020e7ffe8ce423a8bdefec667161217"
format_stamp: "Formatted at 2024-05-02 07:58:16 on dist-test-slave-twcr"
I20240502 07:58:16.975778 6072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:17.002333 6072 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:17.002813 6072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:17.002944 6072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:17.003418 6072 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:17.004182 6072 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:17.004227 6072 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:17.004273 6072 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:17.004300 6072 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:17.011855 6072 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.69:39399
I20240502 07:58:17.011879 6199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.69:39399 every 8 connection(s)
I20240502 07:58:17.012607 6072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240502 07:58:17.014245 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 6072
I20240502 07:58:17.014339 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20240502 07:58:17.016319 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.70:0
--local_ip_for_outbound_sockets=127.3.139.70
--webserver_interface=127.3.139.70
--webserver_port=0
--tserver_master_addrs=127.3.139.126:34225
--builtin_ntp_servers=127.3.139.84:41319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 07:58:17.020702 6200 heartbeater.cc:347] Connected to a master server at 127.3.139.126:34225
I20240502 07:58:17.020893 6200 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:17.021358 6200 heartbeater.cc:511] Master 127.3.139.126:34225 requested a full tablet report, sending...
I20240502 07:58:17.022230 5488 ts_manager.cc:194] Registered new tserver with Master: e020e7ffe8ce423a8bdefec667161217 (127.3.139.69:39399)
I20240502 07:58:17.022742 5488 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.69:44377
W20240502 07:58:17.113971 6203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:17.114128 6203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:17.114169 6203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:17.117038 6203 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:58:17.117090 6203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:17.117168 6203 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.70
I20240502 07:58:17.120560 6203 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:41319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.70:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.3.139.70
--webserver_port=0
--tserver_master_addrs=127.3.139.126:34225
--never_fsync=true
--heap_profile_path=/tmp/kudu.6203
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.70
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:17.121255 6203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:17.122171 6203 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:17.129045 6209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:17.129369 6208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:17.129747 6211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:17.130182 6203 server_base.cc:1034] running on GCE node
I20240502 07:58:17.130477 6203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:17.130913 6203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:17.132052 6203 hybrid_clock.cc:648] HybridClock initialized: now 1714636697132044 us; error 28 us; skew 500 ppm
I20240502 07:58:17.133598 6203 webserver.cc:460] Webserver started at http://127.3.139.70:37079/ using document root <none> and password file <none>
I20240502 07:58:17.134048 6203 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:17.134096 6203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:17.134250 6203 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:17.135653 6203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/data/instance:
uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb"
format_stamp: "Formatted at 2024-05-02 07:58:17 on dist-test-slave-twcr"
I20240502 07:58:17.136060 6203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/wal/instance:
uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb"
format_stamp: "Formatted at 2024-05-02 07:58:17 on dist-test-slave-twcr"
I20240502 07:58:17.138763 6203 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20240502 07:58:17.140687 6217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:17.141539 6203 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20240502 07:58:17.141621 6203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb"
format_stamp: "Formatted at 2024-05-02 07:58:17 on dist-test-slave-twcr"
I20240502 07:58:17.141693 6203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:17.150444 6203 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:17.150897 6203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:17.151024 6203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:17.151492 6203 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:17.152218 6203 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:17.152261 6203 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:17.152297 6203 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:17.152323 6203 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:17.159703 6203 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.70:41047
I20240502 07:58:17.159727 6330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.70:41047 every 8 connection(s)
I20240502 07:58:17.160423 6203 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240502 07:58:17.160714 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 6203
I20240502 07:58:17.160811 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/wal/instance
I20240502 07:58:17.162747 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.71:0
--local_ip_for_outbound_sockets=127.3.139.71
--webserver_interface=127.3.139.71
--webserver_port=0
--tserver_master_addrs=127.3.139.126:34225
--builtin_ntp_servers=127.3.139.84:41319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 07:58:17.168292 6331 heartbeater.cc:347] Connected to a master server at 127.3.139.126:34225
I20240502 07:58:17.168469 6331 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:17.168895 6331 heartbeater.cc:511] Master 127.3.139.126:34225 requested a full tablet report, sending...
I20240502 07:58:17.169875 5488 ts_manager.cc:194] Registered new tserver with Master: d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70:41047)
I20240502 07:58:17.170375 5488 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.70:32811
W20240502 07:58:17.259979 6334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:17.260141 6334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:17.260193 6334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:17.263034 6334 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:58:17.263114 6334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:17.263180 6334 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.71
I20240502 07:58:17.266525 6334 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:41319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.71:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.3.139.71
--webserver_port=0
--tserver_master_addrs=127.3.139.126:34225
--never_fsync=true
--heap_profile_path=/tmp/kudu.6334
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.71
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:17.267220 6334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:17.268153 6334 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:17.275271 6342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:17.275557 6339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:17.275938 6340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:17.276403 6334 server_base.cc:1034] running on GCE node
I20240502 07:58:17.276746 6334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:17.277217 6334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:17.278360 6334 hybrid_clock.cc:648] HybridClock initialized: now 1714636697278347 us; error 31 us; skew 500 ppm
I20240502 07:58:17.280061 6334 webserver.cc:460] Webserver started at http://127.3.139.71:45703/ using document root <none> and password file <none>
I20240502 07:58:17.280527 6334 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:17.280580 6334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:17.280732 6334 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:17.282164 6334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/data/instance:
uuid: "27e3fafad7a24bcda362d2197be87017"
format_stamp: "Formatted at 2024-05-02 07:58:17 on dist-test-slave-twcr"
I20240502 07:58:17.282579 6334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/wal/instance:
uuid: "27e3fafad7a24bcda362d2197be87017"
format_stamp: "Formatted at 2024-05-02 07:58:17 on dist-test-slave-twcr"
I20240502 07:58:17.285486 6334 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20240502 07:58:17.287361 6348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:17.288251 6334 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:17.288338 6334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "27e3fafad7a24bcda362d2197be87017"
format_stamp: "Formatted at 2024-05-02 07:58:17 on dist-test-slave-twcr"
I20240502 07:58:17.288408 6334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:17.301239 6334 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:17.301708 6334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:17.301833 6334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:17.302259 6334 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:17.302984 6334 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:17.303027 6334 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:17.303089 6334 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:17.303117 6334 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:17.310524 6334 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.71:35341
I20240502 07:58:17.310549 6461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.71:35341 every 8 connection(s)
I20240502 07:58:17.311282 6334 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240502 07:58:17.317109 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 6334
I20240502 07:58:17.317222 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/wal/instance
I20240502 07:58:17.318392 6462 heartbeater.cc:347] Connected to a master server at 127.3.139.126:34225
I20240502 07:58:17.318583 6462 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:17.319197 6462 heartbeater.cc:511] Master 127.3.139.126:34225 requested a full tablet report, sending...
I20240502 07:58:17.320001 5487 ts_manager.cc:194] Registered new tserver with Master: 27e3fafad7a24bcda362d2197be87017 (127.3.139.71:35341)
I20240502 07:58:17.320499 5487 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.71:59401
I20240502 07:58:17.330744 3629 external_mini_cluster.cc:930] 7 TS(s) registered with all masters
I20240502 07:58:17.338596 5487 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40652:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 5
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20240502 07:58:17.356089 6003 tablet_service.cc:1467] Processing CreateTablet for tablet cd070443095547d1bf82a6c4a7666b86 (DEFAULT_TABLE table=TestTable [id=ce12803e6d704c758d74d2e1fb8a39b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:17.356084 6265 tablet_service.cc:1467] Processing CreateTablet for tablet cd070443095547d1bf82a6c4a7666b86 (DEFAULT_TABLE table=TestTable [id=ce12803e6d704c758d74d2e1fb8a39b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:17.357008 6265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd070443095547d1bf82a6c4a7666b86. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:17.357007 6003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd070443095547d1bf82a6c4a7666b86. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:17.357918 6134 tablet_service.cc:1467] Processing CreateTablet for tablet cd070443095547d1bf82a6c4a7666b86 (DEFAULT_TABLE table=TestTable [id=ce12803e6d704c758d74d2e1fb8a39b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:17.358652 6134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd070443095547d1bf82a6c4a7666b86. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:17.360241 5741 tablet_service.cc:1467] Processing CreateTablet for tablet cd070443095547d1bf82a6c4a7666b86 (DEFAULT_TABLE table=TestTable [id=ce12803e6d704c758d74d2e1fb8a39b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:17.361030 5741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd070443095547d1bf82a6c4a7666b86. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:17.366344 6486 tablet_bootstrap.cc:492] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d: Bootstrap starting.
I20240502 07:58:17.366154 5608 tablet_service.cc:1467] Processing CreateTablet for tablet cd070443095547d1bf82a6c4a7666b86 (DEFAULT_TABLE table=TestTable [id=ce12803e6d704c758d74d2e1fb8a39b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:17.366873 5608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd070443095547d1bf82a6c4a7666b86. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:17.367012 6484 tablet_bootstrap.cc:492] T cd070443095547d1bf82a6c4a7666b86 P d58f6d1ff8cf472f8a9edf3700e64bfb: Bootstrap starting.
I20240502 07:58:17.367256 6487 tablet_bootstrap.cc:492] T cd070443095547d1bf82a6c4a7666b86 P eda55a8566c447cfa749457825b70688: Bootstrap starting.
I20240502 07:58:17.367552 5676 heartbeater.cc:503] Master 127.3.139.126:34225 was elected leader, sending a full tablet report...
I20240502 07:58:17.368767 6484 tablet_bootstrap.cc:654] T cd070443095547d1bf82a6c4a7666b86 P d58f6d1ff8cf472f8a9edf3700e64bfb: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:17.368767 6486 tablet_bootstrap.cc:654] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:17.369505 6484 log.cc:824] T cd070443095547d1bf82a6c4a7666b86 P d58f6d1ff8cf472f8a9edf3700e64bfb: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:17.369505 6486 log.cc:824] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:17.370085 6485 tablet_bootstrap.cc:492] T cd070443095547d1bf82a6c4a7666b86 P e020e7ffe8ce423a8bdefec667161217: Bootstrap starting.
I20240502 07:58:17.370249 6487 tablet_bootstrap.cc:654] T cd070443095547d1bf82a6c4a7666b86 P eda55a8566c447cfa749457825b70688: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:17.371019 6487 log.cc:824] T cd070443095547d1bf82a6c4a7666b86 P eda55a8566c447cfa749457825b70688: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:17.371701 6486 tablet_bootstrap.cc:492] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d: No bootstrap required, opened a new log
I20240502 07:58:17.371847 6485 tablet_bootstrap.cc:654] T cd070443095547d1bf82a6c4a7666b86 P e020e7ffe8ce423a8bdefec667161217: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:17.371862 6486 ts_tablet_manager.cc:1367] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20240502 07:58:17.372515 6485 log.cc:824] T cd070443095547d1bf82a6c4a7666b86 P e020e7ffe8ce423a8bdefec667161217: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:17.372614 6487 tablet_bootstrap.cc:492] T cd070443095547d1bf82a6c4a7666b86 P eda55a8566c447cfa749457825b70688: No bootstrap required, opened a new log
I20240502 07:58:17.372767 6487 ts_tablet_manager.cc:1367] T cd070443095547d1bf82a6c4a7666b86 P eda55a8566c447cfa749457825b70688: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20240502 07:58:17.373919 6485 tablet_bootstrap.cc:492] T cd070443095547d1bf82a6c4a7666b86 P e020e7ffe8ce423a8bdefec667161217: No bootstrap required, opened a new log
I20240502 07:58:17.374056 6485 ts_tablet_manager.cc:1367] T cd070443095547d1bf82a6c4a7666b86 P e020e7ffe8ce423a8bdefec667161217: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240502 07:58:17.374748 6486 raft_consensus.cc:348] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:17.374954 6486 raft_consensus.cc:374] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:17.375005 6486 raft_consensus.cc:729] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5bc7863cd294f77a932c4a40f98b79d, State: Initialized, Role: FOLLOWER
I20240502 07:58:17.375000 6487 raft_consensus.cc:348] T cd070443095547d1bf82a6c4a7666b86 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:17.375205 6487 raft_consensus.cc:374] T cd070443095547d1bf82a6c4a7666b86 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:17.375253 6487 raft_consensus.cc:729] T cd070443095547d1bf82a6c4a7666b86 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eda55a8566c447cfa749457825b70688, State: Initialized, Role: FOLLOWER
I20240502 07:58:17.375413 6486 consensus_queue.cc:260] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:17.375615 6491 tablet_bootstrap.cc:492] T cd070443095547d1bf82a6c4a7666b86 P 91e82003a656442189a329b29846a841: Bootstrap starting.
I20240502 07:58:17.375934 6487 consensus_queue.cc:260] T cd070443095547d1bf82a6c4a7666b86 P eda55a8566c447cfa749457825b70688 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:17.376490 6484 tablet_bootstrap.cc:492] T cd070443095547d1bf82a6c4a7666b86 P d58f6d1ff8cf472f8a9edf3700e64bfb: No bootstrap required, opened a new log
I20240502 07:58:17.376649 6484 ts_tablet_manager.cc:1367] T cd070443095547d1bf82a6c4a7666b86 P d58f6d1ff8cf472f8a9edf3700e64bfb: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.000s
I20240502 07:58:17.377233 6487 ts_tablet_manager.cc:1398] T cd070443095547d1bf82a6c4a7666b86 P eda55a8566c447cfa749457825b70688: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20240502 07:58:17.377511 6491 tablet_bootstrap.cc:654] T cd070443095547d1bf82a6c4a7666b86 P 91e82003a656442189a329b29846a841: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:17.377609 5807 heartbeater.cc:503] Master 127.3.139.126:34225 was elected leader, sending a full tablet report...
I20240502 07:58:17.378615 6485 raft_consensus.cc:348] T cd070443095547d1bf82a6c4a7666b86 P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:17.378844 6485 raft_consensus.cc:374] T cd070443095547d1bf82a6c4a7666b86 P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:17.378890 6485 raft_consensus.cc:729] T cd070443095547d1bf82a6c4a7666b86 P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e020e7ffe8ce423a8bdefec667161217, State: Initialized, Role: FOLLOWER
I20240502 07:58:17.378911 6491 log.cc:824] T cd070443095547d1bf82a6c4a7666b86 P 91e82003a656442189a329b29846a841: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:17.379284 6485 consensus_queue.cc:260] T cd070443095547d1bf82a6c4a7666b86 P e020e7ffe8ce423a8bdefec667161217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:17.380084 6485 ts_tablet_manager.cc:1398] T cd070443095547d1bf82a6c4a7666b86 P e020e7ffe8ce423a8bdefec667161217: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20240502 07:58:17.380021 6484 raft_consensus.cc:348] T cd070443095547d1bf82a6c4a7666b86 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:17.380192 6484 raft_consensus.cc:374] T cd070443095547d1bf82a6c4a7666b86 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:17.380231 6484 raft_consensus.cc:729] T cd070443095547d1bf82a6c4a7666b86 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d58f6d1ff8cf472f8a9edf3700e64bfb, State: Initialized, Role: FOLLOWER
I20240502 07:58:17.380374 6200 heartbeater.cc:503] Master 127.3.139.126:34225 was elected leader, sending a full tablet report...
I20240502 07:58:17.380595 6484 consensus_queue.cc:260] T cd070443095547d1bf82a6c4a7666b86 P d58f6d1ff8cf472f8a9edf3700e64bfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:17.380849 6491 tablet_bootstrap.cc:492] T cd070443095547d1bf82a6c4a7666b86 P 91e82003a656442189a329b29846a841: No bootstrap required, opened a new log
I20240502 07:58:17.380993 6491 ts_tablet_manager.cc:1367] T cd070443095547d1bf82a6c4a7666b86 P 91e82003a656442189a329b29846a841: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20240502 07:58:17.381325 6484 ts_tablet_manager.cc:1398] T cd070443095547d1bf82a6c4a7666b86 P d58f6d1ff8cf472f8a9edf3700e64bfb: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20240502 07:58:17.381646 6331 heartbeater.cc:503] Master 127.3.139.126:34225 was elected leader, sending a full tablet report...
I20240502 07:58:17.382093 6486 ts_tablet_manager.cc:1398] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d: Time spent starting tablet: real 0.010s user 0.005s sys 0.000s
I20240502 07:58:17.382405 6069 heartbeater.cc:503] Master 127.3.139.126:34225 was elected leader, sending a full tablet report...
I20240502 07:58:17.384292 6491 raft_consensus.cc:348] T cd070443095547d1bf82a6c4a7666b86 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:17.384479 6491 raft_consensus.cc:374] T cd070443095547d1bf82a6c4a7666b86 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:17.384542 6491 raft_consensus.cc:729] T cd070443095547d1bf82a6c4a7666b86 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91e82003a656442189a329b29846a841, State: Initialized, Role: FOLLOWER
I20240502 07:58:17.384907 6491 consensus_queue.cc:260] T cd070443095547d1bf82a6c4a7666b86 P 91e82003a656442189a329b29846a841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:17.385618 6491 ts_tablet_manager.cc:1398] T cd070443095547d1bf82a6c4a7666b86 P 91e82003a656442189a329b29846a841: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20240502 07:58:17.411157 6332 tablet.cc:2369] T cd070443095547d1bf82a6c4a7666b86 P d58f6d1ff8cf472f8a9edf3700e64bfb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 07:58:17.428649 6497 raft_consensus.cc:482] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:17.428768 6497 raft_consensus.cc:504] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:17.429742 6497 leader_election.cc:288] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70:41047), eda55a8566c447cfa749457825b70688 (127.3.139.66:39925), e020e7ffe8ce423a8bdefec667161217 (127.3.139.69:39399), 91e82003a656442189a329b29846a841 (127.3.139.65:43429)
I20240502 07:58:17.434249 6493 raft_consensus.cc:482] T cd070443095547d1bf82a6c4a7666b86 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:17.434401 6493 raft_consensus.cc:504] T cd070443095547d1bf82a6c4a7666b86 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:17.434778 5761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd070443095547d1bf82a6c4a7666b86" candidate_uuid: "d5bc7863cd294f77a932c4a40f98b79d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eda55a8566c447cfa749457825b70688" is_pre_election: true
I20240502 07:58:17.434851 6285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd070443095547d1bf82a6c4a7666b86" candidate_uuid: "d5bc7863cd294f77a932c4a40f98b79d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" is_pre_election: true
I20240502 07:58:17.434991 5761 raft_consensus.cc:2454] T cd070443095547d1bf82a6c4a7666b86 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5bc7863cd294f77a932c4a40f98b79d in term 0.
I20240502 07:58:17.435091 6285 raft_consensus.cc:2454] T cd070443095547d1bf82a6c4a7666b86 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5bc7863cd294f77a932c4a40f98b79d in term 0.
I20240502 07:58:17.435415 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd070443095547d1bf82a6c4a7666b86" candidate_uuid: "d5bc7863cd294f77a932c4a40f98b79d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e82003a656442189a329b29846a841" is_pre_election: true
I20240502 07:58:17.435627 5630 raft_consensus.cc:2454] T cd070443095547d1bf82a6c4a7666b86 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5bc7863cd294f77a932c4a40f98b79d in term 0.
I20240502 07:58:17.435652 6154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd070443095547d1bf82a6c4a7666b86" candidate_uuid: "d5bc7863cd294f77a932c4a40f98b79d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e020e7ffe8ce423a8bdefec667161217" is_pre_election: true
I20240502 07:58:17.435777 5958 leader_election.cc:302] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: d58f6d1ff8cf472f8a9edf3700e64bfb, d5bc7863cd294f77a932c4a40f98b79d, eda55a8566c447cfa749457825b70688; no voters:
I20240502 07:58:17.435870 6154 raft_consensus.cc:2454] T cd070443095547d1bf82a6c4a7666b86 P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5bc7863cd294f77a932c4a40f98b79d in term 0.
I20240502 07:58:17.436008 6497 raft_consensus.cc:2781] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:17.436069 6497 raft_consensus.cc:482] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:17.436111 6497 raft_consensus.cc:3037] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:17.437120 6497 raft_consensus.cc:504] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:17.437575 6497 leader_election.cc:288] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [CANDIDATE]: Term 1 election: Requested vote from peers d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70:41047), eda55a8566c447cfa749457825b70688 (127.3.139.66:39925), e020e7ffe8ce423a8bdefec667161217 (127.3.139.69:39399), 91e82003a656442189a329b29846a841 (127.3.139.65:43429)
I20240502 07:58:17.437812 6285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd070443095547d1bf82a6c4a7666b86" candidate_uuid: "d5bc7863cd294f77a932c4a40f98b79d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb"
I20240502 07:58:17.437855 6493 leader_election.cc:288] T cd070443095547d1bf82a6c4a7666b86 P eda55a8566c447cfa749457825b70688 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68:44287), d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70:41047), e020e7ffe8ce423a8bdefec667161217 (127.3.139.69:39399), 91e82003a656442189a329b29846a841 (127.3.139.65:43429)
I20240502 07:58:17.437925 6285 raft_consensus.cc:3037] T cd070443095547d1bf82a6c4a7666b86 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:17.437935 6154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd070443095547d1bf82a6c4a7666b86" candidate_uuid: "d5bc7863cd294f77a932c4a40f98b79d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e020e7ffe8ce423a8bdefec667161217"
I20240502 07:58:17.438040 6154 raft_consensus.cc:3037] T cd070443095547d1bf82a6c4a7666b86 P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:17.438119 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd070443095547d1bf82a6c4a7666b86" candidate_uuid: "d5bc7863cd294f77a932c4a40f98b79d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e82003a656442189a329b29846a841"
I20240502 07:58:17.438201 5630 raft_consensus.cc:3037] T cd070443095547d1bf82a6c4a7666b86 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:17.438886 6285 raft_consensus.cc:2454] T cd070443095547d1bf82a6c4a7666b86 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5bc7863cd294f77a932c4a40f98b79d in term 1.
I20240502 07:58:17.439075 6154 raft_consensus.cc:2454] T cd070443095547d1bf82a6c4a7666b86 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5bc7863cd294f77a932c4a40f98b79d in term 1.
I20240502 07:58:17.439139 5630 raft_consensus.cc:2454] T cd070443095547d1bf82a6c4a7666b86 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5bc7863cd294f77a932c4a40f98b79d in term 1.
I20240502 07:58:17.439440 5957 leader_election.cc:302] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 91e82003a656442189a329b29846a841, d58f6d1ff8cf472f8a9edf3700e64bfb, d5bc7863cd294f77a932c4a40f98b79d; no voters:
I20240502 07:58:17.439745 5761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd070443095547d1bf82a6c4a7666b86" candidate_uuid: "d5bc7863cd294f77a932c4a40f98b79d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eda55a8566c447cfa749457825b70688"
I20240502 07:58:17.439842 5761 raft_consensus.cc:3037] T cd070443095547d1bf82a6c4a7666b86 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:17.440766 5761 raft_consensus.cc:2454] T cd070443095547d1bf82a6c4a7666b86 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5bc7863cd294f77a932c4a40f98b79d in term 1.
I20240502 07:58:17.441273 6497 raft_consensus.cc:2781] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:17.441462 6497 raft_consensus.cc:686] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [term 1 LEADER]: Becoming Leader. State: Replica: d5bc7863cd294f77a932c4a40f98b79d, State: Running, Role: LEADER
I20240502 07:58:17.441634 6023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd070443095547d1bf82a6c4a7666b86" candidate_uuid: "eda55a8566c447cfa749457825b70688" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5bc7863cd294f77a932c4a40f98b79d" is_pre_election: true
I20240502 07:58:17.441752 6497 consensus_queue.cc:237] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:17.442940 6285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd070443095547d1bf82a6c4a7666b86" candidate_uuid: "eda55a8566c447cfa749457825b70688" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" is_pre_election: true
I20240502 07:58:17.443097 6285 raft_consensus.cc:2379] T cd070443095547d1bf82a6c4a7666b86 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eda55a8566c447cfa749457825b70688 in current term 1: Already voted for candidate d5bc7863cd294f77a932c4a40f98b79d in this term.
I20240502 07:58:17.443136 6154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd070443095547d1bf82a6c4a7666b86" candidate_uuid: "eda55a8566c447cfa749457825b70688" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e020e7ffe8ce423a8bdefec667161217" is_pre_election: true
I20240502 07:58:17.443269 6154 raft_consensus.cc:2379] T cd070443095547d1bf82a6c4a7666b86 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eda55a8566c447cfa749457825b70688 in current term 1: Already voted for candidate d5bc7863cd294f77a932c4a40f98b79d in this term.
I20240502 07:58:17.443610 5696 leader_election.cc:302] T cd070443095547d1bf82a6c4a7666b86 P eda55a8566c447cfa749457825b70688 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 5 voters: 1 yes votes; 3 no votes. yes voters: eda55a8566c447cfa749457825b70688; no voters: d58f6d1ff8cf472f8a9edf3700e64bfb, d5bc7863cd294f77a932c4a40f98b79d, e020e7ffe8ce423a8bdefec667161217
I20240502 07:58:17.443802 6493 raft_consensus.cc:2726] T cd070443095547d1bf82a6c4a7666b86 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 07:58:17.444561 5487 catalog_manager.cc:5484] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d reported cstate change: term changed from 0 to 1, leader changed from <none> to d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68). New cstate: current_term: 1 leader_uuid: "d5bc7863cd294f77a932c4a40f98b79d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:17.445325 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd070443095547d1bf82a6c4a7666b86" candidate_uuid: "eda55a8566c447cfa749457825b70688" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e82003a656442189a329b29846a841" is_pre_election: true
I20240502 07:58:17.445461 5630 raft_consensus.cc:2379] T cd070443095547d1bf82a6c4a7666b86 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eda55a8566c447cfa749457825b70688 in current term 1: Already voted for candidate d5bc7863cd294f77a932c4a40f98b79d in this term.
I20240502 07:58:17.452895 3629 external_mini_cluster.cc:930] 7 TS(s) registered with all masters
I20240502 07:58:17.454090 3629 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 91e82003a656442189a329b29846a841 to finish bootstrapping
I20240502 07:58:17.458669 3629 ts_itest-base.cc:233] Waiting for 1 tablets on tserver eda55a8566c447cfa749457825b70688 to finish bootstrapping
I20240502 07:58:17.462558 3629 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 89bf5c0a6b2a4d949851089446f17b96 to finish bootstrapping
I20240502 07:58:17.467463 3629 ts_itest-base.cc:233] Waiting for 1 tablets on tserver d5bc7863cd294f77a932c4a40f98b79d to finish bootstrapping
I20240502 07:58:17.471606 3629 ts_itest-base.cc:233] Waiting for 1 tablets on tserver e020e7ffe8ce423a8bdefec667161217 to finish bootstrapping
I20240502 07:58:17.475598 3629 ts_itest-base.cc:233] Waiting for 1 tablets on tserver d58f6d1ff8cf472f8a9edf3700e64bfb to finish bootstrapping
I20240502 07:58:17.479624 3629 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 27e3fafad7a24bcda362d2197be87017 to finish bootstrapping
I20240502 07:58:17.484602 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 6334
W20240502 07:58:17.513549 6201 tablet.cc:2369] T cd070443095547d1bf82a6c4a7666b86 P e020e7ffe8ce423a8bdefec667161217: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240502 07:58:17.517791 5808 tablet.cc:2369] T cd070443095547d1bf82a6c4a7666b86 P eda55a8566c447cfa749457825b70688: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240502 07:58:17.593632 6070 tablet.cc:2369] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240502 07:58:17.604748 5677 tablet.cc:2369] T cd070443095547d1bf82a6c4a7666b86 P 91e82003a656442189a329b29846a841: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 07:58:17.694883 5938 heartbeater.cc:503] Master 127.3.139.126:34225 was elected leader, sending a full tablet report...
I20240502 07:58:17.863530 6497 consensus_queue.cc:1035] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:17.866190 6497 consensus_queue.cc:1035] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:17.871583 6516 consensus_queue.cc:1035] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:17.872287 6516 consensus_queue.cc:1035] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:21.240193 5487 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40652:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 5
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:58:21.246590 6003 tablet_service.cc:1467] Processing CreateTablet for tablet d14545ba1e474960909a1b140b6a315c (DEFAULT_TABLE table=rebalance_test_table_0 [id=e1e32c7b8b8c4c05b2f82c3195606811]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:21.246927 6003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d14545ba1e474960909a1b140b6a315c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:21.247476 6134 tablet_service.cc:1467] Processing CreateTablet for tablet d14545ba1e474960909a1b140b6a315c (DEFAULT_TABLE table=rebalance_test_table_0 [id=e1e32c7b8b8c4c05b2f82c3195606811]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:21.247473 5608 tablet_service.cc:1467] Processing CreateTablet for tablet d14545ba1e474960909a1b140b6a315c (DEFAULT_TABLE table=rebalance_test_table_0 [id=e1e32c7b8b8c4c05b2f82c3195606811]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:21.247746 6134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d14545ba1e474960909a1b140b6a315c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:21.247766 5608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d14545ba1e474960909a1b140b6a315c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:21.248627 5740 tablet_service.cc:1467] Processing CreateTablet for tablet 5b87554a301044caac956bc5c7f137e8 (DEFAULT_TABLE table=rebalance_test_table_0 [id=e1e32c7b8b8c4c05b2f82c3195606811]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:21.248919 5740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b87554a301044caac956bc5c7f137e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:21.249431 6528 tablet_bootstrap.cc:492] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d: Bootstrap starting.
I20240502 07:58:21.250038 6002 tablet_service.cc:1467] Processing CreateTablet for tablet 5b87554a301044caac956bc5c7f137e8 (DEFAULT_TABLE table=rebalance_test_table_0 [id=e1e32c7b8b8c4c05b2f82c3195606811]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:21.250030 5741 tablet_service.cc:1467] Processing CreateTablet for tablet d14545ba1e474960909a1b140b6a315c (DEFAULT_TABLE table=rebalance_test_table_0 [id=e1e32c7b8b8c4c05b2f82c3195606811]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:21.250255 5741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d14545ba1e474960909a1b140b6a315c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:21.250254 6002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b87554a301044caac956bc5c7f137e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:21.250308 6528 tablet_bootstrap.cc:654] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:21.251143 6532 tablet_bootstrap.cc:492] T 5b87554a301044caac956bc5c7f137e8 P eda55a8566c447cfa749457825b70688: Bootstrap starting.
I20240502 07:58:21.251278 5739 tablet_service.cc:1467] Processing CreateTablet for tablet f1a11f8d412949218b8af5de7c63505f (DEFAULT_TABLE table=rebalance_test_table_0 [id=e1e32c7b8b8c4c05b2f82c3195606811]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:21.251498 5739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1a11f8d412949218b8af5de7c63505f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:21.251986 6532 tablet_bootstrap.cc:654] T 5b87554a301044caac956bc5c7f137e8 P eda55a8566c447cfa749457825b70688: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:21.252349 6530 tablet_bootstrap.cc:492] T d14545ba1e474960909a1b140b6a315c P e020e7ffe8ce423a8bdefec667161217: Bootstrap starting.
I20240502 07:58:21.253167 6265 tablet_service.cc:1467] Processing CreateTablet for tablet 5b87554a301044caac956bc5c7f137e8 (DEFAULT_TABLE table=rebalance_test_table_0 [id=e1e32c7b8b8c4c05b2f82c3195606811]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:21.253437 6265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b87554a301044caac956bc5c7f137e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:21.253526 6134 tablet_service.cc:1467] Processing CreateTablet for tablet f1a11f8d412949218b8af5de7c63505f (DEFAULT_TABLE table=rebalance_test_table_0 [id=e1e32c7b8b8c4c05b2f82c3195606811]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:21.253719 6134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1a11f8d412949218b8af5de7c63505f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:21.253759 5608 tablet_service.cc:1467] Processing CreateTablet for tablet 5b87554a301044caac956bc5c7f137e8 (DEFAULT_TABLE table=rebalance_test_table_0 [id=e1e32c7b8b8c4c05b2f82c3195606811]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:21.253968 5608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b87554a301044caac956bc5c7f137e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:21.255849 5610 tablet_service.cc:1467] Processing CreateTablet for tablet f1a11f8d412949218b8af5de7c63505f (DEFAULT_TABLE table=rebalance_test_table_0 [id=e1e32c7b8b8c4c05b2f82c3195606811]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:21.256103 5610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1a11f8d412949218b8af5de7c63505f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:21.256207 6535 tablet_bootstrap.cc:492] T 5b87554a301044caac956bc5c7f137e8 P d58f6d1ff8cf472f8a9edf3700e64bfb: Bootstrap starting.
I20240502 07:58:21.256340 6530 tablet_bootstrap.cc:654] T d14545ba1e474960909a1b140b6a315c P e020e7ffe8ce423a8bdefec667161217: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:21.257048 6535 tablet_bootstrap.cc:654] T 5b87554a301044caac956bc5c7f137e8 P d58f6d1ff8cf472f8a9edf3700e64bfb: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:21.253180 6264 tablet_service.cc:1467] Processing CreateTablet for tablet f1a11f8d412949218b8af5de7c63505f (DEFAULT_TABLE table=rebalance_test_table_0 [id=e1e32c7b8b8c4c05b2f82c3195606811]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:21.258147 6264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1a11f8d412949218b8af5de7c63505f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:21.256974 6529 tablet_bootstrap.cc:492] T d14545ba1e474960909a1b140b6a315c P 91e82003a656442189a329b29846a841: Bootstrap starting.
I20240502 07:58:21.258589 6535 tablet_bootstrap.cc:492] T 5b87554a301044caac956bc5c7f137e8 P d58f6d1ff8cf472f8a9edf3700e64bfb: No bootstrap required, opened a new log
I20240502 07:58:21.258659 6535 ts_tablet_manager.cc:1367] T 5b87554a301044caac956bc5c7f137e8 P d58f6d1ff8cf472f8a9edf3700e64bfb: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:58:21.259285 6529 tablet_bootstrap.cc:654] T d14545ba1e474960909a1b140b6a315c P 91e82003a656442189a329b29846a841: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:21.260387 6528 tablet_bootstrap.cc:492] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d: No bootstrap required, opened a new log
I20240502 07:58:21.260466 6528 ts_tablet_manager.cc:1367] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20240502 07:58:21.260926 6528 raft_consensus.cc:348] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:21.261121 6528 raft_consensus.cc:374] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:21.261200 6528 raft_consensus.cc:729] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5bc7863cd294f77a932c4a40f98b79d, State: Initialized, Role: FOLLOWER
I20240502 07:58:21.261391 6528 consensus_queue.cc:260] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:21.261737 6528 ts_tablet_manager.cc:1398] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:21.261410 5872 tablet_service.cc:1467] Processing CreateTablet for tablet d14545ba1e474960909a1b140b6a315c (DEFAULT_TABLE table=rebalance_test_table_0 [id=e1e32c7b8b8c4c05b2f82c3195606811]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:21.262050 6528 tablet_bootstrap.cc:492] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d: Bootstrap starting.
I20240502 07:58:21.262027 5870 tablet_service.cc:1467] Processing CreateTablet for tablet f1a11f8d412949218b8af5de7c63505f (DEFAULT_TABLE table=rebalance_test_table_0 [id=e1e32c7b8b8c4c05b2f82c3195606811]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:21.262375 5872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d14545ba1e474960909a1b140b6a315c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:21.262267 6535 raft_consensus.cc:348] T 5b87554a301044caac956bc5c7f137e8 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:21.262419 6535 raft_consensus.cc:374] T 5b87554a301044caac956bc5c7f137e8 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:21.262473 6535 raft_consensus.cc:729] T 5b87554a301044caac956bc5c7f137e8 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d58f6d1ff8cf472f8a9edf3700e64bfb, State: Initialized, Role: FOLLOWER
I20240502 07:58:21.262714 5870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1a11f8d412949218b8af5de7c63505f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:21.262778 6528 tablet_bootstrap.cc:654] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:21.261408 5871 tablet_service.cc:1467] Processing CreateTablet for tablet 5b87554a301044caac956bc5c7f137e8 (DEFAULT_TABLE table=rebalance_test_table_0 [id=e1e32c7b8b8c4c05b2f82c3195606811]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:21.263576 6532 tablet_bootstrap.cc:492] T 5b87554a301044caac956bc5c7f137e8 P eda55a8566c447cfa749457825b70688: No bootstrap required, opened a new log
I20240502 07:58:21.263612 5871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b87554a301044caac956bc5c7f137e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:21.263650 6532 ts_tablet_manager.cc:1367] T 5b87554a301044caac956bc5c7f137e8 P eda55a8566c447cfa749457825b70688: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.000s
I20240502 07:58:21.264161 6532 raft_consensus.cc:348] T 5b87554a301044caac956bc5c7f137e8 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:21.264330 6532 raft_consensus.cc:374] T 5b87554a301044caac956bc5c7f137e8 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:21.264376 6532 raft_consensus.cc:729] T 5b87554a301044caac956bc5c7f137e8 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eda55a8566c447cfa749457825b70688, State: Initialized, Role: FOLLOWER
I20240502 07:58:21.264494 6532 consensus_queue.cc:260] T 5b87554a301044caac956bc5c7f137e8 P eda55a8566c447cfa749457825b70688 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:21.264833 6532 ts_tablet_manager.cc:1398] T 5b87554a301044caac956bc5c7f137e8 P eda55a8566c447cfa749457825b70688: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:21.265041 6532 tablet_bootstrap.cc:492] T d14545ba1e474960909a1b140b6a315c P eda55a8566c447cfa749457825b70688: Bootstrap starting.
I20240502 07:58:21.265874 6532 tablet_bootstrap.cc:654] T d14545ba1e474960909a1b140b6a315c P eda55a8566c447cfa749457825b70688: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:21.266801 6532 tablet_bootstrap.cc:492] T d14545ba1e474960909a1b140b6a315c P eda55a8566c447cfa749457825b70688: No bootstrap required, opened a new log
I20240502 07:58:21.266877 6532 ts_tablet_manager.cc:1367] T d14545ba1e474960909a1b140b6a315c P eda55a8566c447cfa749457825b70688: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:21.267376 6528 tablet_bootstrap.cc:492] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d: No bootstrap required, opened a new log
I20240502 07:58:21.267450 6528 ts_tablet_manager.cc:1367] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240502 07:58:21.267803 6528 raft_consensus.cc:348] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:21.267913 6528 raft_consensus.cc:374] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:21.267946 6528 raft_consensus.cc:729] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5bc7863cd294f77a932c4a40f98b79d, State: Initialized, Role: FOLLOWER
I20240502 07:58:21.268056 6528 consensus_queue.cc:260] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:21.268272 6528 ts_tablet_manager.cc:1398] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:21.268577 6532 raft_consensus.cc:348] T d14545ba1e474960909a1b140b6a315c P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:21.268663 6532 raft_consensus.cc:374] T d14545ba1e474960909a1b140b6a315c P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:21.268690 6532 raft_consensus.cc:729] T d14545ba1e474960909a1b140b6a315c P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eda55a8566c447cfa749457825b70688, State: Initialized, Role: FOLLOWER
I20240502 07:58:21.268815 6532 consensus_queue.cc:260] T d14545ba1e474960909a1b140b6a315c P eda55a8566c447cfa749457825b70688 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:21.269102 6532 ts_tablet_manager.cc:1398] T d14545ba1e474960909a1b140b6a315c P eda55a8566c447cfa749457825b70688: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:21.269152 6542 tablet_bootstrap.cc:492] T d14545ba1e474960909a1b140b6a315c P 89bf5c0a6b2a4d949851089446f17b96: Bootstrap starting.
I20240502 07:58:21.269291 6532 tablet_bootstrap.cc:492] T f1a11f8d412949218b8af5de7c63505f P eda55a8566c447cfa749457825b70688: Bootstrap starting.
I20240502 07:58:21.270114 6532 tablet_bootstrap.cc:654] T f1a11f8d412949218b8af5de7c63505f P eda55a8566c447cfa749457825b70688: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:21.270115 6535 consensus_queue.cc:260] T 5b87554a301044caac956bc5c7f137e8 P d58f6d1ff8cf472f8a9edf3700e64bfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:21.270531 6535 ts_tablet_manager.cc:1398] T 5b87554a301044caac956bc5c7f137e8 P d58f6d1ff8cf472f8a9edf3700e64bfb: Time spent starting tablet: real 0.012s user 0.001s sys 0.000s
I20240502 07:58:21.271100 6532 tablet_bootstrap.cc:492] T f1a11f8d412949218b8af5de7c63505f P eda55a8566c447cfa749457825b70688: No bootstrap required, opened a new log
I20240502 07:58:21.271180 6532 ts_tablet_manager.cc:1367] T f1a11f8d412949218b8af5de7c63505f P eda55a8566c447cfa749457825b70688: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:21.271298 6542 tablet_bootstrap.cc:654] T d14545ba1e474960909a1b140b6a315c P 89bf5c0a6b2a4d949851089446f17b96: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:21.271560 6532 raft_consensus.cc:348] T f1a11f8d412949218b8af5de7c63505f P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } }
I20240502 07:58:21.271647 6532 raft_consensus.cc:374] T f1a11f8d412949218b8af5de7c63505f P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:21.271674 6532 raft_consensus.cc:729] T f1a11f8d412949218b8af5de7c63505f P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eda55a8566c447cfa749457825b70688, State: Initialized, Role: FOLLOWER
I20240502 07:58:21.271770 6532 consensus_queue.cc:260] T f1a11f8d412949218b8af5de7c63505f P eda55a8566c447cfa749457825b70688 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } }
I20240502 07:58:21.271968 6535 tablet_bootstrap.cc:492] T f1a11f8d412949218b8af5de7c63505f P d58f6d1ff8cf472f8a9edf3700e64bfb: Bootstrap starting.
I20240502 07:58:21.272078 6532 ts_tablet_manager.cc:1398] T f1a11f8d412949218b8af5de7c63505f P eda55a8566c447cfa749457825b70688: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:21.272295 6542 log.cc:824] T d14545ba1e474960909a1b140b6a315c P 89bf5c0a6b2a4d949851089446f17b96: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:21.272763 6535 tablet_bootstrap.cc:654] T f1a11f8d412949218b8af5de7c63505f P d58f6d1ff8cf472f8a9edf3700e64bfb: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:21.273941 6542 tablet_bootstrap.cc:492] T d14545ba1e474960909a1b140b6a315c P 89bf5c0a6b2a4d949851089446f17b96: No bootstrap required, opened a new log
I20240502 07:58:21.274032 6535 tablet_bootstrap.cc:492] T f1a11f8d412949218b8af5de7c63505f P d58f6d1ff8cf472f8a9edf3700e64bfb: No bootstrap required, opened a new log
I20240502 07:58:21.274091 6535 ts_tablet_manager.cc:1367] T f1a11f8d412949218b8af5de7c63505f P d58f6d1ff8cf472f8a9edf3700e64bfb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:21.274101 6542 ts_tablet_manager.cc:1367] T d14545ba1e474960909a1b140b6a315c P 89bf5c0a6b2a4d949851089446f17b96: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20240502 07:58:21.274145 6530 tablet_bootstrap.cc:492] T d14545ba1e474960909a1b140b6a315c P e020e7ffe8ce423a8bdefec667161217: No bootstrap required, opened a new log
I20240502 07:58:21.274226 6530 ts_tablet_manager.cc:1367] T d14545ba1e474960909a1b140b6a315c P e020e7ffe8ce423a8bdefec667161217: Time spent bootstrapping tablet: real 0.022s user 0.001s sys 0.000s
I20240502 07:58:21.274487 6535 raft_consensus.cc:348] T f1a11f8d412949218b8af5de7c63505f P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } }
I20240502 07:58:21.274574 6535 raft_consensus.cc:374] T f1a11f8d412949218b8af5de7c63505f P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:21.274602 6535 raft_consensus.cc:729] T f1a11f8d412949218b8af5de7c63505f P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d58f6d1ff8cf472f8a9edf3700e64bfb, State: Initialized, Role: FOLLOWER
I20240502 07:58:21.274621 6530 raft_consensus.cc:348] T d14545ba1e474960909a1b140b6a315c P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:21.274711 6530 raft_consensus.cc:374] T d14545ba1e474960909a1b140b6a315c P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:21.274741 6530 raft_consensus.cc:729] T d14545ba1e474960909a1b140b6a315c P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e020e7ffe8ce423a8bdefec667161217, State: Initialized, Role: FOLLOWER
I20240502 07:58:21.274708 6535 consensus_queue.cc:260] T f1a11f8d412949218b8af5de7c63505f P d58f6d1ff8cf472f8a9edf3700e64bfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } }
I20240502 07:58:21.274847 6530 consensus_queue.cc:260] T d14545ba1e474960909a1b140b6a315c P e020e7ffe8ce423a8bdefec667161217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:21.275017 6535 ts_tablet_manager.cc:1398] T f1a11f8d412949218b8af5de7c63505f P d58f6d1ff8cf472f8a9edf3700e64bfb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:21.275169 6530 ts_tablet_manager.cc:1398] T d14545ba1e474960909a1b140b6a315c P e020e7ffe8ce423a8bdefec667161217: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:21.275300 6530 tablet_bootstrap.cc:492] T f1a11f8d412949218b8af5de7c63505f P e020e7ffe8ce423a8bdefec667161217: Bootstrap starting.
I20240502 07:58:21.276122 6530 tablet_bootstrap.cc:654] T f1a11f8d412949218b8af5de7c63505f P e020e7ffe8ce423a8bdefec667161217: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:21.277380 6530 tablet_bootstrap.cc:492] T f1a11f8d412949218b8af5de7c63505f P e020e7ffe8ce423a8bdefec667161217: No bootstrap required, opened a new log
I20240502 07:58:21.277468 6530 ts_tablet_manager.cc:1367] T f1a11f8d412949218b8af5de7c63505f P e020e7ffe8ce423a8bdefec667161217: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:21.277474 6529 tablet_bootstrap.cc:492] T d14545ba1e474960909a1b140b6a315c P 91e82003a656442189a329b29846a841: No bootstrap required, opened a new log
I20240502 07:58:21.277552 6529 ts_tablet_manager.cc:1367] T d14545ba1e474960909a1b140b6a315c P 91e82003a656442189a329b29846a841: Time spent bootstrapping tablet: real 0.021s user 0.001s sys 0.000s
I20240502 07:58:21.277868 6530 raft_consensus.cc:348] T f1a11f8d412949218b8af5de7c63505f P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } }
I20240502 07:58:21.277985 6530 raft_consensus.cc:374] T f1a11f8d412949218b8af5de7c63505f P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:21.278019 6530 raft_consensus.cc:729] T f1a11f8d412949218b8af5de7c63505f P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e020e7ffe8ce423a8bdefec667161217, State: Initialized, Role: FOLLOWER
I20240502 07:58:21.277998 6529 raft_consensus.cc:348] T d14545ba1e474960909a1b140b6a315c P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:21.278199 6530 consensus_queue.cc:260] T f1a11f8d412949218b8af5de7c63505f P e020e7ffe8ce423a8bdefec667161217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } }
I20240502 07:58:21.278445 6530 ts_tablet_manager.cc:1398] T f1a11f8d412949218b8af5de7c63505f P e020e7ffe8ce423a8bdefec667161217: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:21.278937 6542 raft_consensus.cc:348] T d14545ba1e474960909a1b140b6a315c P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:21.279215 6529 raft_consensus.cc:374] T d14545ba1e474960909a1b140b6a315c P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:21.279278 6529 raft_consensus.cc:729] T d14545ba1e474960909a1b140b6a315c P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91e82003a656442189a329b29846a841, State: Initialized, Role: FOLLOWER
I20240502 07:58:21.279403 6529 consensus_queue.cc:260] T d14545ba1e474960909a1b140b6a315c P 91e82003a656442189a329b29846a841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:21.279738 6529 ts_tablet_manager.cc:1398] T d14545ba1e474960909a1b140b6a315c P 91e82003a656442189a329b29846a841: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:21.279913 6529 tablet_bootstrap.cc:492] T 5b87554a301044caac956bc5c7f137e8 P 91e82003a656442189a329b29846a841: Bootstrap starting.
I20240502 07:58:21.281309 6529 tablet_bootstrap.cc:654] T 5b87554a301044caac956bc5c7f137e8 P 91e82003a656442189a329b29846a841: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:21.280000 6542 raft_consensus.cc:374] T d14545ba1e474960909a1b140b6a315c P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:21.281425 6542 raft_consensus.cc:729] T d14545ba1e474960909a1b140b6a315c P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89bf5c0a6b2a4d949851089446f17b96, State: Initialized, Role: FOLLOWER
I20240502 07:58:21.281877 6542 consensus_queue.cc:260] T d14545ba1e474960909a1b140b6a315c P 89bf5c0a6b2a4d949851089446f17b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:21.282536 6542 ts_tablet_manager.cc:1398] T d14545ba1e474960909a1b140b6a315c P 89bf5c0a6b2a4d949851089446f17b96: Time spent starting tablet: real 0.008s user 0.004s sys 0.000s
I20240502 07:58:21.283129 6542 tablet_bootstrap.cc:492] T 5b87554a301044caac956bc5c7f137e8 P 89bf5c0a6b2a4d949851089446f17b96: Bootstrap starting.
I20240502 07:58:21.284058 6542 tablet_bootstrap.cc:654] T 5b87554a301044caac956bc5c7f137e8 P 89bf5c0a6b2a4d949851089446f17b96: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:21.284833 6529 tablet_bootstrap.cc:492] T 5b87554a301044caac956bc5c7f137e8 P 91e82003a656442189a329b29846a841: No bootstrap required, opened a new log
I20240502 07:58:21.284920 6529 ts_tablet_manager.cc:1367] T 5b87554a301044caac956bc5c7f137e8 P 91e82003a656442189a329b29846a841: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:58:21.285245 6529 raft_consensus.cc:348] T 5b87554a301044caac956bc5c7f137e8 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:21.285324 6529 raft_consensus.cc:374] T 5b87554a301044caac956bc5c7f137e8 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:21.285351 6529 raft_consensus.cc:729] T 5b87554a301044caac956bc5c7f137e8 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91e82003a656442189a329b29846a841, State: Initialized, Role: FOLLOWER
I20240502 07:58:21.285477 6529 consensus_queue.cc:260] T 5b87554a301044caac956bc5c7f137e8 P 91e82003a656442189a329b29846a841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:21.285686 6529 ts_tablet_manager.cc:1398] T 5b87554a301044caac956bc5c7f137e8 P 91e82003a656442189a329b29846a841: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:21.285825 6529 tablet_bootstrap.cc:492] T f1a11f8d412949218b8af5de7c63505f P 91e82003a656442189a329b29846a841: Bootstrap starting.
I20240502 07:58:21.286504 6529 tablet_bootstrap.cc:654] T f1a11f8d412949218b8af5de7c63505f P 91e82003a656442189a329b29846a841: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:21.287305 6542 tablet_bootstrap.cc:492] T 5b87554a301044caac956bc5c7f137e8 P 89bf5c0a6b2a4d949851089446f17b96: No bootstrap required, opened a new log
I20240502 07:58:21.287381 6542 ts_tablet_manager.cc:1367] T 5b87554a301044caac956bc5c7f137e8 P 89bf5c0a6b2a4d949851089446f17b96: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 07:58:21.287760 6542 raft_consensus.cc:348] T 5b87554a301044caac956bc5c7f137e8 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:21.287897 6542 raft_consensus.cc:374] T 5b87554a301044caac956bc5c7f137e8 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:21.287946 6542 raft_consensus.cc:729] T 5b87554a301044caac956bc5c7f137e8 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89bf5c0a6b2a4d949851089446f17b96, State: Initialized, Role: FOLLOWER
I20240502 07:58:21.288044 6542 consensus_queue.cc:260] T 5b87554a301044caac956bc5c7f137e8 P 89bf5c0a6b2a4d949851089446f17b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:21.288228 6542 ts_tablet_manager.cc:1398] T 5b87554a301044caac956bc5c7f137e8 P 89bf5c0a6b2a4d949851089446f17b96: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:21.288547 6529 tablet_bootstrap.cc:492] T f1a11f8d412949218b8af5de7c63505f P 91e82003a656442189a329b29846a841: No bootstrap required, opened a new log
I20240502 07:58:21.288616 6529 ts_tablet_manager.cc:1367] T f1a11f8d412949218b8af5de7c63505f P 91e82003a656442189a329b29846a841: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:58:21.288934 6529 raft_consensus.cc:348] T f1a11f8d412949218b8af5de7c63505f P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } }
I20240502 07:58:21.289011 6529 raft_consensus.cc:374] T f1a11f8d412949218b8af5de7c63505f P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:21.289038 6529 raft_consensus.cc:729] T f1a11f8d412949218b8af5de7c63505f P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91e82003a656442189a329b29846a841, State: Initialized, Role: FOLLOWER
I20240502 07:58:21.289142 6529 consensus_queue.cc:260] T f1a11f8d412949218b8af5de7c63505f P 91e82003a656442189a329b29846a841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } }
I20240502 07:58:21.289359 6529 ts_tablet_manager.cc:1398] T f1a11f8d412949218b8af5de7c63505f P 91e82003a656442189a329b29846a841: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:21.289522 6542 tablet_bootstrap.cc:492] T f1a11f8d412949218b8af5de7c63505f P 89bf5c0a6b2a4d949851089446f17b96: Bootstrap starting.
I20240502 07:58:21.290225 6542 tablet_bootstrap.cc:654] T f1a11f8d412949218b8af5de7c63505f P 89bf5c0a6b2a4d949851089446f17b96: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:21.291273 6542 tablet_bootstrap.cc:492] T f1a11f8d412949218b8af5de7c63505f P 89bf5c0a6b2a4d949851089446f17b96: No bootstrap required, opened a new log
I20240502 07:58:21.291361 6542 ts_tablet_manager.cc:1367] T f1a11f8d412949218b8af5de7c63505f P 89bf5c0a6b2a4d949851089446f17b96: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:21.291760 6542 raft_consensus.cc:348] T f1a11f8d412949218b8af5de7c63505f P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } }
I20240502 07:58:21.291898 6542 raft_consensus.cc:374] T f1a11f8d412949218b8af5de7c63505f P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:21.291955 6542 raft_consensus.cc:729] T f1a11f8d412949218b8af5de7c63505f P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89bf5c0a6b2a4d949851089446f17b96, State: Initialized, Role: FOLLOWER
I20240502 07:58:21.292116 6542 consensus_queue.cc:260] T f1a11f8d412949218b8af5de7c63505f P 89bf5c0a6b2a4d949851089446f17b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } }
I20240502 07:58:21.292371 6542 ts_tablet_manager.cc:1398] T f1a11f8d412949218b8af5de7c63505f P 89bf5c0a6b2a4d949851089446f17b96: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:21.352654 6540 raft_consensus.cc:482] T f1a11f8d412949218b8af5de7c63505f P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:21.352769 6540 raft_consensus.cc:504] T f1a11f8d412949218b8af5de7c63505f P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } }
I20240502 07:58:21.353349 6540 leader_election.cc:288] T f1a11f8d412949218b8af5de7c63505f P eda55a8566c447cfa749457825b70688 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70:41047), e020e7ffe8ce423a8bdefec667161217 (127.3.139.69:39399), 91e82003a656442189a329b29846a841 (127.3.139.65:43429), 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67:37141)
I20240502 07:58:21.353700 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1a11f8d412949218b8af5de7c63505f" candidate_uuid: "eda55a8566c447cfa749457825b70688" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e82003a656442189a329b29846a841" is_pre_election: true
I20240502 07:58:21.353839 5630 raft_consensus.cc:2454] T f1a11f8d412949218b8af5de7c63505f P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eda55a8566c447cfa749457825b70688 in term 0.
I20240502 07:58:21.353816 6285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1a11f8d412949218b8af5de7c63505f" candidate_uuid: "eda55a8566c447cfa749457825b70688" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" is_pre_election: true
I20240502 07:58:21.353945 6285 raft_consensus.cc:2454] T f1a11f8d412949218b8af5de7c63505f P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eda55a8566c447cfa749457825b70688 in term 0.
I20240502 07:58:21.354192 6154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1a11f8d412949218b8af5de7c63505f" candidate_uuid: "eda55a8566c447cfa749457825b70688" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e020e7ffe8ce423a8bdefec667161217" is_pre_election: true
I20240502 07:58:21.354329 6154 raft_consensus.cc:2454] T f1a11f8d412949218b8af5de7c63505f P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eda55a8566c447cfa749457825b70688 in term 0.
I20240502 07:58:21.354323 5696 leader_election.cc:302] T f1a11f8d412949218b8af5de7c63505f P eda55a8566c447cfa749457825b70688 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 91e82003a656442189a329b29846a841, d58f6d1ff8cf472f8a9edf3700e64bfb, eda55a8566c447cfa749457825b70688; no voters:
I20240502 07:58:21.354552 6540 raft_consensus.cc:2781] T f1a11f8d412949218b8af5de7c63505f P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:21.354626 6540 raft_consensus.cc:482] T f1a11f8d412949218b8af5de7c63505f P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:21.354684 6540 raft_consensus.cc:3037] T f1a11f8d412949218b8af5de7c63505f P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:21.355599 6540 raft_consensus.cc:504] T f1a11f8d412949218b8af5de7c63505f P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } }
I20240502 07:58:21.356003 6540 leader_election.cc:288] T f1a11f8d412949218b8af5de7c63505f P eda55a8566c447cfa749457825b70688 [CANDIDATE]: Term 1 election: Requested vote from peers d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70:41047), e020e7ffe8ce423a8bdefec667161217 (127.3.139.69:39399), 91e82003a656442189a329b29846a841 (127.3.139.65:43429), 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67:37141)
I20240502 07:58:21.356292 6285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1a11f8d412949218b8af5de7c63505f" candidate_uuid: "eda55a8566c447cfa749457825b70688" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb"
I20240502 07:58:21.356402 6285 raft_consensus.cc:3037] T f1a11f8d412949218b8af5de7c63505f P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:21.356380 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1a11f8d412949218b8af5de7c63505f" candidate_uuid: "eda55a8566c447cfa749457825b70688" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e82003a656442189a329b29846a841"
I20240502 07:58:21.356421 6154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1a11f8d412949218b8af5de7c63505f" candidate_uuid: "eda55a8566c447cfa749457825b70688" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e020e7ffe8ce423a8bdefec667161217"
I20240502 07:58:21.356472 5630 raft_consensus.cc:3037] T f1a11f8d412949218b8af5de7c63505f P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:21.356506 6154 raft_consensus.cc:3037] T f1a11f8d412949218b8af5de7c63505f P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:21.357299 6285 raft_consensus.cc:2454] T f1a11f8d412949218b8af5de7c63505f P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eda55a8566c447cfa749457825b70688 in term 1.
I20240502 07:58:21.357306 5630 raft_consensus.cc:2454] T f1a11f8d412949218b8af5de7c63505f P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eda55a8566c447cfa749457825b70688 in term 1.
I20240502 07:58:21.357306 6154 raft_consensus.cc:2454] T f1a11f8d412949218b8af5de7c63505f P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eda55a8566c447cfa749457825b70688 in term 1.
I20240502 07:58:21.357689 5696 leader_election.cc:302] T f1a11f8d412949218b8af5de7c63505f P eda55a8566c447cfa749457825b70688 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 91e82003a656442189a329b29846a841, e020e7ffe8ce423a8bdefec667161217, eda55a8566c447cfa749457825b70688; no voters:
I20240502 07:58:21.357827 6540 raft_consensus.cc:2781] T f1a11f8d412949218b8af5de7c63505f P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:21.357941 6540 raft_consensus.cc:686] T f1a11f8d412949218b8af5de7c63505f P eda55a8566c447cfa749457825b70688 [term 1 LEADER]: Becoming Leader. State: Replica: eda55a8566c447cfa749457825b70688, State: Running, Role: LEADER
I20240502 07:58:21.358249 6540 consensus_queue.cc:237] T f1a11f8d412949218b8af5de7c63505f P eda55a8566c447cfa749457825b70688 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } }
I20240502 07:58:21.360222 5892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1a11f8d412949218b8af5de7c63505f" candidate_uuid: "eda55a8566c447cfa749457825b70688" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89bf5c0a6b2a4d949851089446f17b96" is_pre_election: true
I20240502 07:58:21.360494 5892 raft_consensus.cc:2454] T f1a11f8d412949218b8af5de7c63505f P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eda55a8566c447cfa749457825b70688 in term 0.
I20240502 07:58:21.360785 5891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1a11f8d412949218b8af5de7c63505f" candidate_uuid: "eda55a8566c447cfa749457825b70688" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89bf5c0a6b2a4d949851089446f17b96"
I20240502 07:58:21.360919 5891 raft_consensus.cc:3037] T f1a11f8d412949218b8af5de7c63505f P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:21.361652 5891 raft_consensus.cc:2454] T f1a11f8d412949218b8af5de7c63505f P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eda55a8566c447cfa749457825b70688 in term 1.
I20240502 07:58:21.362196 5488 catalog_manager.cc:5484] T f1a11f8d412949218b8af5de7c63505f P eda55a8566c447cfa749457825b70688 reported cstate change: term changed from 0 to 1, leader changed from <none> to eda55a8566c447cfa749457825b70688 (127.3.139.66). New cstate: current_term: 1 leader_uuid: "eda55a8566c447cfa749457825b70688" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:21.411541 6515 raft_consensus.cc:482] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:21.411640 6515 raft_consensus.cc:504] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:21.412134 6515 leader_election.cc:288] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70:41047), 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67:37141), eda55a8566c447cfa749457825b70688 (127.3.139.66:39925), 91e82003a656442189a329b29846a841 (127.3.139.65:43429)
I20240502 07:58:21.412441 6285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b87554a301044caac956bc5c7f137e8" candidate_uuid: "d5bc7863cd294f77a932c4a40f98b79d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" is_pre_election: true
I20240502 07:58:21.412573 6285 raft_consensus.cc:2454] T 5b87554a301044caac956bc5c7f137e8 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5bc7863cd294f77a932c4a40f98b79d in term 0.
I20240502 07:58:21.412837 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b87554a301044caac956bc5c7f137e8" candidate_uuid: "d5bc7863cd294f77a932c4a40f98b79d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e82003a656442189a329b29846a841" is_pre_election: true
I20240502 07:58:21.412839 5761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b87554a301044caac956bc5c7f137e8" candidate_uuid: "d5bc7863cd294f77a932c4a40f98b79d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eda55a8566c447cfa749457825b70688" is_pre_election: true
I20240502 07:58:21.412956 5761 raft_consensus.cc:2454] T 5b87554a301044caac956bc5c7f137e8 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5bc7863cd294f77a932c4a40f98b79d in term 0.
I20240502 07:58:21.412959 5630 raft_consensus.cc:2454] T 5b87554a301044caac956bc5c7f137e8 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5bc7863cd294f77a932c4a40f98b79d in term 0.
I20240502 07:58:21.413259 5957 leader_election.cc:302] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 91e82003a656442189a329b29846a841, d58f6d1ff8cf472f8a9edf3700e64bfb, d5bc7863cd294f77a932c4a40f98b79d; no voters:
I20240502 07:58:21.413434 6515 raft_consensus.cc:2781] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:21.413502 6515 raft_consensus.cc:482] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:21.413540 6515 raft_consensus.cc:3037] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:21.414374 6515 raft_consensus.cc:504] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:21.414844 6515 leader_election.cc:288] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d [CANDIDATE]: Term 1 election: Requested vote from peers d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70:41047), 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67:37141), eda55a8566c447cfa749457825b70688 (127.3.139.66:39925), 91e82003a656442189a329b29846a841 (127.3.139.65:43429)
I20240502 07:58:21.415041 6285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b87554a301044caac956bc5c7f137e8" candidate_uuid: "d5bc7863cd294f77a932c4a40f98b79d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb"
I20240502 07:58:21.415175 6285 raft_consensus.cc:3037] T 5b87554a301044caac956bc5c7f137e8 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:21.415158 5761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b87554a301044caac956bc5c7f137e8" candidate_uuid: "d5bc7863cd294f77a932c4a40f98b79d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eda55a8566c447cfa749457825b70688"
I20240502 07:58:21.415187 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b87554a301044caac956bc5c7f137e8" candidate_uuid: "d5bc7863cd294f77a932c4a40f98b79d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e82003a656442189a329b29846a841"
I20240502 07:58:21.415269 5761 raft_consensus.cc:3037] T 5b87554a301044caac956bc5c7f137e8 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:21.415272 5630 raft_consensus.cc:3037] T 5b87554a301044caac956bc5c7f137e8 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:21.416033 6285 raft_consensus.cc:2454] T 5b87554a301044caac956bc5c7f137e8 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5bc7863cd294f77a932c4a40f98b79d in term 1.
I20240502 07:58:21.416033 5630 raft_consensus.cc:2454] T 5b87554a301044caac956bc5c7f137e8 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5bc7863cd294f77a932c4a40f98b79d in term 1.
I20240502 07:58:21.416175 5761 raft_consensus.cc:2454] T 5b87554a301044caac956bc5c7f137e8 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5bc7863cd294f77a932c4a40f98b79d in term 1.
I20240502 07:58:21.416354 5957 leader_election.cc:302] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 91e82003a656442189a329b29846a841, d58f6d1ff8cf472f8a9edf3700e64bfb, d5bc7863cd294f77a932c4a40f98b79d; no voters:
I20240502 07:58:21.416507 6515 raft_consensus.cc:2781] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:21.416697 6515 raft_consensus.cc:686] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d [term 1 LEADER]: Becoming Leader. State: Replica: d5bc7863cd294f77a932c4a40f98b79d, State: Running, Role: LEADER
I20240502 07:58:21.416759 5891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b87554a301044caac956bc5c7f137e8" candidate_uuid: "d5bc7863cd294f77a932c4a40f98b79d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89bf5c0a6b2a4d949851089446f17b96" is_pre_election: true
I20240502 07:58:21.416802 5892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b87554a301044caac956bc5c7f137e8" candidate_uuid: "d5bc7863cd294f77a932c4a40f98b79d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89bf5c0a6b2a4d949851089446f17b96"
I20240502 07:58:21.416868 5891 raft_consensus.cc:2454] T 5b87554a301044caac956bc5c7f137e8 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5bc7863cd294f77a932c4a40f98b79d in term 0.
I20240502 07:58:21.416843 6515 consensus_queue.cc:237] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:21.418394 5488 catalog_manager.cc:5484] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d reported cstate change: term changed from 0 to 1, leader changed from <none> to d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68). New cstate: current_term: 1 leader_uuid: "d5bc7863cd294f77a932c4a40f98b79d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: UNKNOWN } } }
W20240502 07:58:21.437430 5939 tablet.cc:2369] T 5b87554a301044caac956bc5c7f137e8 P 89bf5c0a6b2a4d949851089446f17b96: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 07:58:21.450193 6515 raft_consensus.cc:482] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:21.450275 6515 raft_consensus.cc:504] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:21.450627 6515 leader_election.cc:288] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67:37141), eda55a8566c447cfa749457825b70688 (127.3.139.66:39925), e020e7ffe8ce423a8bdefec667161217 (127.3.139.69:39399), 91e82003a656442189a329b29846a841 (127.3.139.65:43429)
I20240502 07:58:21.450878 5892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d14545ba1e474960909a1b140b6a315c" candidate_uuid: "d5bc7863cd294f77a932c4a40f98b79d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89bf5c0a6b2a4d949851089446f17b96" is_pre_election: true
I20240502 07:58:21.450963 5761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d14545ba1e474960909a1b140b6a315c" candidate_uuid: "d5bc7863cd294f77a932c4a40f98b79d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eda55a8566c447cfa749457825b70688" is_pre_election: true
I20240502 07:58:21.450964 6154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d14545ba1e474960909a1b140b6a315c" candidate_uuid: "d5bc7863cd294f77a932c4a40f98b79d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e020e7ffe8ce423a8bdefec667161217" is_pre_election: true
I20240502 07:58:21.450997 5892 raft_consensus.cc:2454] T d14545ba1e474960909a1b140b6a315c P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5bc7863cd294f77a932c4a40f98b79d in term 0.
I20240502 07:58:21.450968 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d14545ba1e474960909a1b140b6a315c" candidate_uuid: "d5bc7863cd294f77a932c4a40f98b79d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e82003a656442189a329b29846a841" is_pre_election: true
I20240502 07:58:21.451081 5761 raft_consensus.cc:2454] T d14545ba1e474960909a1b140b6a315c P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5bc7863cd294f77a932c4a40f98b79d in term 0.
I20240502 07:58:21.451088 6154 raft_consensus.cc:2454] T d14545ba1e474960909a1b140b6a315c P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5bc7863cd294f77a932c4a40f98b79d in term 0.
I20240502 07:58:21.451107 5630 raft_consensus.cc:2454] T d14545ba1e474960909a1b140b6a315c P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5bc7863cd294f77a932c4a40f98b79d in term 0.
I20240502 07:58:21.451359 5957 leader_election.cc:302] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 89bf5c0a6b2a4d949851089446f17b96, d5bc7863cd294f77a932c4a40f98b79d, e020e7ffe8ce423a8bdefec667161217; no voters:
I20240502 07:58:21.451507 6515 raft_consensus.cc:2781] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:21.451557 6515 raft_consensus.cc:482] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:21.451601 6515 raft_consensus.cc:3037] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:21.452260 6515 raft_consensus.cc:504] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:21.452649 6515 leader_election.cc:288] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [CANDIDATE]: Term 1 election: Requested vote from peers 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67:37141), eda55a8566c447cfa749457825b70688 (127.3.139.66:39925), e020e7ffe8ce423a8bdefec667161217 (127.3.139.69:39399), 91e82003a656442189a329b29846a841 (127.3.139.65:43429)
I20240502 07:58:21.452912 5892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d14545ba1e474960909a1b140b6a315c" candidate_uuid: "d5bc7863cd294f77a932c4a40f98b79d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89bf5c0a6b2a4d949851089446f17b96"
I20240502 07:58:21.452922 5761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d14545ba1e474960909a1b140b6a315c" candidate_uuid: "d5bc7863cd294f77a932c4a40f98b79d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eda55a8566c447cfa749457825b70688"
I20240502 07:58:21.452957 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d14545ba1e474960909a1b140b6a315c" candidate_uuid: "d5bc7863cd294f77a932c4a40f98b79d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e82003a656442189a329b29846a841"
I20240502 07:58:21.452965 6154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d14545ba1e474960909a1b140b6a315c" candidate_uuid: "d5bc7863cd294f77a932c4a40f98b79d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e020e7ffe8ce423a8bdefec667161217"
I20240502 07:58:21.453007 5892 raft_consensus.cc:3037] T d14545ba1e474960909a1b140b6a315c P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:21.453007 5761 raft_consensus.cc:3037] T d14545ba1e474960909a1b140b6a315c P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:21.453045 5630 raft_consensus.cc:3037] T d14545ba1e474960909a1b140b6a315c P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:21.453075 6154 raft_consensus.cc:3037] T d14545ba1e474960909a1b140b6a315c P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:21.453819 5892 raft_consensus.cc:2454] T d14545ba1e474960909a1b140b6a315c P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5bc7863cd294f77a932c4a40f98b79d in term 1.
I20240502 07:58:21.453814 5761 raft_consensus.cc:2454] T d14545ba1e474960909a1b140b6a315c P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5bc7863cd294f77a932c4a40f98b79d in term 1.
I20240502 07:58:21.453863 5630 raft_consensus.cc:2454] T d14545ba1e474960909a1b140b6a315c P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5bc7863cd294f77a932c4a40f98b79d in term 1.
I20240502 07:58:21.453872 6154 raft_consensus.cc:2454] T d14545ba1e474960909a1b140b6a315c P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5bc7863cd294f77a932c4a40f98b79d in term 1.
I20240502 07:58:21.454169 5957 leader_election.cc:302] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 89bf5c0a6b2a4d949851089446f17b96, d5bc7863cd294f77a932c4a40f98b79d, e020e7ffe8ce423a8bdefec667161217; no voters:
I20240502 07:58:21.454331 6515 raft_consensus.cc:2781] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:21.454442 6515 raft_consensus.cc:686] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [term 1 LEADER]: Becoming Leader. State: Replica: d5bc7863cd294f77a932c4a40f98b79d, State: Running, Role: LEADER
I20240502 07:58:21.454579 6515 consensus_queue.cc:237] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:21.456082 5488 catalog_manager.cc:5484] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d reported cstate change: term changed from 0 to 1, leader changed from <none> to d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68). New cstate: current_term: 1 leader_uuid: "d5bc7863cd294f77a932c4a40f98b79d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: UNKNOWN } } }
W20240502 07:58:21.582127 6563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:21.582306 6563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:21.585289 6563 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_0'
I20240502 07:58:21.658598 5630 raft_consensus.cc:1261] T d14545ba1e474960909a1b140b6a315c P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:21.659145 6154 raft_consensus.cc:1261] T d14545ba1e474960909a1b140b6a315c P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:21.659237 6285 raft_consensus.cc:1261] T 5b87554a301044caac956bc5c7f137e8 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:21.659480 5761 raft_consensus.cc:1261] T 5b87554a301044caac956bc5c7f137e8 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:21.659642 5892 raft_consensus.cc:3037] T 5b87554a301044caac956bc5c7f137e8 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:21.659777 6560 consensus_queue.cc:1035] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:21.660064 6560 consensus_queue.cc:1035] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:21.660300 6560 consensus_queue.cc:1035] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:21.660547 6560 consensus_queue.cc:1035] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:21.660887 5892 raft_consensus.cc:1261] T 5b87554a301044caac956bc5c7f137e8 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:21.660979 5630 raft_consensus.cc:1261] T 5b87554a301044caac956bc5c7f137e8 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:21.662142 6560 consensus_queue.cc:1035] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:21.662420 6560 consensus_queue.cc:1035] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:21.662472 5892 raft_consensus.cc:1261] T d14545ba1e474960909a1b140b6a315c P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:21.663903 5891 raft_consensus.cc:1261] T f1a11f8d412949218b8af5de7c63505f P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer eda55a8566c447cfa749457825b70688: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:21.664297 6284 raft_consensus.cc:1261] T f1a11f8d412949218b8af5de7c63505f P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Refusing update from remote peer eda55a8566c447cfa749457825b70688: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:21.664623 5629 raft_consensus.cc:1261] T f1a11f8d412949218b8af5de7c63505f P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer eda55a8566c447cfa749457825b70688: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:21.665211 5760 raft_consensus.cc:1261] T d14545ba1e474960909a1b140b6a315c P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:21.665549 6545 consensus_queue.cc:1035] T f1a11f8d412949218b8af5de7c63505f P eda55a8566c447cfa749457825b70688 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:21.665619 6598 consensus_queue.cc:1035] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:21.668723 6545 consensus_queue.cc:1035] T f1a11f8d412949218b8af5de7c63505f P eda55a8566c447cfa749457825b70688 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:21.669308 6540 consensus_queue.cc:1035] T f1a11f8d412949218b8af5de7c63505f P eda55a8566c447cfa749457825b70688 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:21.670364 6515 consensus_queue.cc:1035] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:21.672791 6154 raft_consensus.cc:1261] T f1a11f8d412949218b8af5de7c63505f P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer eda55a8566c447cfa749457825b70688: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:21.673133 6540 consensus_queue.cc:1035] T f1a11f8d412949218b8af5de7c63505f P eda55a8566c447cfa749457825b70688 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:21.688329 6594 mvcc.cc:205] Tried to move back new op lower bound from 7023151930005090304 to 7023151929005215744. Current Snapshot: MvccSnapshot[applied={T|T < 7023151929991245824}]
I20240502 07:58:21.693742 6606 mvcc.cc:205] Tried to move back new op lower bound from 7023151930005090304 to 7023151929005215744. Current Snapshot: MvccSnapshot[applied={T|T < 7023151929991245824}]
I20240502 07:58:21.697836 6596 mvcc.cc:205] Tried to move back new op lower bound from 7023151930009624576 to 7023151928768036864. Current Snapshot: MvccSnapshot[applied={T|T < 7023151930009624576}]
I20240502 07:58:21.703409 6619 mvcc.cc:205] Tried to move back new op lower bound from 7023151929989181440 to 7023151929159802880. Current Snapshot: MvccSnapshot[applied={T|T < 7023151929989181440}]
I20240502 07:58:21.713811 6613 mvcc.cc:205] Tried to move back new op lower bound from 7023151930023976960 to 7023151928768036864. Current Snapshot: MvccSnapshot[applied={T|T < 7023151930009624576}]
I20240502 07:58:21.719214 6601 mvcc.cc:205] Tried to move back new op lower bound from 7023151930005090304 to 7023151929005215744. Current Snapshot: MvccSnapshot[applied={T|T < 7023151929991245824}]
INSERT report
rows total: 2000
time total: 144.286 ms
time per row: 0.072143 ms
I20240502 07:58:21.770764 5488 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40652:
name: "rebalance_test_table_1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 5
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:58:21.775686 6002 tablet_service.cc:1467] Processing CreateTablet for tablet d41a2cb7b69e40d9a681384e5ad0a451 (DEFAULT_TABLE table=rebalance_test_table_1 [id=6f5d44f6fbe544099b33ad94e35cee6e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:21.775686 5872 tablet_service.cc:1467] Processing CreateTablet for tablet d41a2cb7b69e40d9a681384e5ad0a451 (DEFAULT_TABLE table=rebalance_test_table_1 [id=6f5d44f6fbe544099b33ad94e35cee6e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:21.775955 5872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d41a2cb7b69e40d9a681384e5ad0a451. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:21.775955 6002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d41a2cb7b69e40d9a681384e5ad0a451. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:21.777060 6265 tablet_service.cc:1467] Processing CreateTablet for tablet acf2ff61e8ff450ab7fca71fb4191dd6 (DEFAULT_TABLE table=rebalance_test_table_1 [id=6f5d44f6fbe544099b33ad94e35cee6e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:21.777060 6264 tablet_service.cc:1467] Processing CreateTablet for tablet d41a2cb7b69e40d9a681384e5ad0a451 (DEFAULT_TABLE table=rebalance_test_table_1 [id=6f5d44f6fbe544099b33ad94e35cee6e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:21.777341 6265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acf2ff61e8ff450ab7fca71fb4191dd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:21.777508 6264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d41a2cb7b69e40d9a681384e5ad0a451. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:21.778043 6542 tablet_bootstrap.cc:492] T d41a2cb7b69e40d9a681384e5ad0a451 P 89bf5c0a6b2a4d949851089446f17b96: Bootstrap starting.
I20240502 07:58:21.778504 6134 tablet_service.cc:1467] Processing CreateTablet for tablet d41a2cb7b69e40d9a681384e5ad0a451 (DEFAULT_TABLE table=rebalance_test_table_1 [id=6f5d44f6fbe544099b33ad94e35cee6e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:21.778736 6134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d41a2cb7b69e40d9a681384e5ad0a451. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:21.778801 6542 tablet_bootstrap.cc:654] T d41a2cb7b69e40d9a681384e5ad0a451 P 89bf5c0a6b2a4d949851089446f17b96: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:21.779413 6132 tablet_service.cc:1467] Processing CreateTablet for tablet 7001a52697f1417bb84a0e3ed673ed0c (DEFAULT_TABLE table=rebalance_test_table_1 [id=6f5d44f6fbe544099b33ad94e35cee6e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:21.779632 6132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7001a52697f1417bb84a0e3ed673ed0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:21.780383 6650 tablet_bootstrap.cc:492] T acf2ff61e8ff450ab7fca71fb4191dd6 P d58f6d1ff8cf472f8a9edf3700e64bfb: Bootstrap starting.
I20240502 07:58:21.780678 6651 tablet_bootstrap.cc:492] T d41a2cb7b69e40d9a681384e5ad0a451 P e020e7ffe8ce423a8bdefec667161217: Bootstrap starting.
I20240502 07:58:21.780854 6133 tablet_service.cc:1467] Processing CreateTablet for tablet acf2ff61e8ff450ab7fca71fb4191dd6 (DEFAULT_TABLE table=rebalance_test_table_1 [id=6f5d44f6fbe544099b33ad94e35cee6e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:21.781088 6133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acf2ff61e8ff450ab7fca71fb4191dd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:21.781216 6650 tablet_bootstrap.cc:654] T acf2ff61e8ff450ab7fca71fb4191dd6 P d58f6d1ff8cf472f8a9edf3700e64bfb: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:21.781468 6263 tablet_service.cc:1467] Processing CreateTablet for tablet 7001a52697f1417bb84a0e3ed673ed0c (DEFAULT_TABLE table=rebalance_test_table_1 [id=6f5d44f6fbe544099b33ad94e35cee6e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:21.781706 6263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7001a52697f1417bb84a0e3ed673ed0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:21.781726 6651 tablet_bootstrap.cc:654] T d41a2cb7b69e40d9a681384e5ad0a451 P e020e7ffe8ce423a8bdefec667161217: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:21.782155 6650 tablet_bootstrap.cc:492] T acf2ff61e8ff450ab7fca71fb4191dd6 P d58f6d1ff8cf472f8a9edf3700e64bfb: No bootstrap required, opened a new log
I20240502 07:58:21.782236 6650 ts_tablet_manager.cc:1367] T acf2ff61e8ff450ab7fca71fb4191dd6 P d58f6d1ff8cf472f8a9edf3700e64bfb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:21.782675 6650 raft_consensus.cc:348] T acf2ff61e8ff450ab7fca71fb4191dd6 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } }
I20240502 07:58:21.782793 6650 raft_consensus.cc:374] T acf2ff61e8ff450ab7fca71fb4191dd6 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:21.782826 6650 raft_consensus.cc:729] T acf2ff61e8ff450ab7fca71fb4191dd6 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d58f6d1ff8cf472f8a9edf3700e64bfb, State: Initialized, Role: FOLLOWER
I20240502 07:58:21.782950 6650 consensus_queue.cc:260] T acf2ff61e8ff450ab7fca71fb4191dd6 P d58f6d1ff8cf472f8a9edf3700e64bfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } }
I20240502 07:58:21.783274 6650 ts_tablet_manager.cc:1398] T acf2ff61e8ff450ab7fca71fb4191dd6 P d58f6d1ff8cf472f8a9edf3700e64bfb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:21.783463 6650 tablet_bootstrap.cc:492] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb: Bootstrap starting.
I20240502 07:58:21.783521 5740 tablet_service.cc:1467] Processing CreateTablet for tablet acf2ff61e8ff450ab7fca71fb4191dd6 (DEFAULT_TABLE table=rebalance_test_table_1 [id=6f5d44f6fbe544099b33ad94e35cee6e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:21.783771 5740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acf2ff61e8ff450ab7fca71fb4191dd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:21.784225 6650 tablet_bootstrap.cc:654] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:21.784508 6003 tablet_service.cc:1467] Processing CreateTablet for tablet 7001a52697f1417bb84a0e3ed673ed0c (DEFAULT_TABLE table=rebalance_test_table_1 [id=6f5d44f6fbe544099b33ad94e35cee6e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:21.784557 6002 tablet_service.cc:1467] Processing CreateTablet for tablet acf2ff61e8ff450ab7fca71fb4191dd6 (DEFAULT_TABLE table=rebalance_test_table_1 [id=6f5d44f6fbe544099b33ad94e35cee6e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:21.784749 6003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7001a52697f1417bb84a0e3ed673ed0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:21.784924 6002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acf2ff61e8ff450ab7fca71fb4191dd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:21.785677 6650 tablet_bootstrap.cc:492] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb: No bootstrap required, opened a new log
I20240502 07:58:21.785755 6650 ts_tablet_manager.cc:1367] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:21.786130 6650 raft_consensus.cc:348] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } }
I20240502 07:58:21.786221 6650 raft_consensus.cc:374] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:21.786250 6650 raft_consensus.cc:729] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d58f6d1ff8cf472f8a9edf3700e64bfb, State: Initialized, Role: FOLLOWER
I20240502 07:58:21.786357 6650 consensus_queue.cc:260] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } }
I20240502 07:58:21.786466 5608 tablet_service.cc:1467] Processing CreateTablet for tablet 7001a52697f1417bb84a0e3ed673ed0c (DEFAULT_TABLE table=rebalance_test_table_1 [id=6f5d44f6fbe544099b33ad94e35cee6e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:21.786599 6650 ts_tablet_manager.cc:1398] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:21.786706 5608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7001a52697f1417bb84a0e3ed673ed0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:21.786733 6650 tablet_bootstrap.cc:492] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb: Bootstrap starting.
I20240502 07:58:21.787176 6542 tablet_bootstrap.cc:492] T d41a2cb7b69e40d9a681384e5ad0a451 P 89bf5c0a6b2a4d949851089446f17b96: No bootstrap required, opened a new log
I20240502 07:58:21.787245 6542 ts_tablet_manager.cc:1367] T d41a2cb7b69e40d9a681384e5ad0a451 P 89bf5c0a6b2a4d949851089446f17b96: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.001s
I20240502 07:58:21.787505 5610 tablet_service.cc:1467] Processing CreateTablet for tablet d41a2cb7b69e40d9a681384e5ad0a451 (DEFAULT_TABLE table=rebalance_test_table_1 [id=6f5d44f6fbe544099b33ad94e35cee6e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:21.787577 6542 raft_consensus.cc:348] T d41a2cb7b69e40d9a681384e5ad0a451 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } }
I20240502 07:58:21.787645 6650 tablet_bootstrap.cc:654] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:21.787676 6542 raft_consensus.cc:374] T d41a2cb7b69e40d9a681384e5ad0a451 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:21.787708 6542 raft_consensus.cc:729] T d41a2cb7b69e40d9a681384e5ad0a451 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89bf5c0a6b2a4d949851089446f17b96, State: Initialized, Role: FOLLOWER
I20240502 07:58:21.787714 5610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d41a2cb7b69e40d9a681384e5ad0a451. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:21.787803 6542 consensus_queue.cc:260] T d41a2cb7b69e40d9a681384e5ad0a451 P 89bf5c0a6b2a4d949851089446f17b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } }
I20240502 07:58:21.788053 6542 ts_tablet_manager.cc:1398] T d41a2cb7b69e40d9a681384e5ad0a451 P 89bf5c0a6b2a4d949851089446f17b96: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:21.788578 6529 tablet_bootstrap.cc:492] T 7001a52697f1417bb84a0e3ed673ed0c P 91e82003a656442189a329b29846a841: Bootstrap starting.
I20240502 07:58:21.789332 6529 tablet_bootstrap.cc:654] T 7001a52697f1417bb84a0e3ed673ed0c P 91e82003a656442189a329b29846a841: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:21.789718 5870 tablet_service.cc:1467] Processing CreateTablet for tablet 7001a52697f1417bb84a0e3ed673ed0c (DEFAULT_TABLE table=rebalance_test_table_1 [id=6f5d44f6fbe544099b33ad94e35cee6e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:21.789932 5870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7001a52697f1417bb84a0e3ed673ed0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:21.790163 6650 tablet_bootstrap.cc:492] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb: No bootstrap required, opened a new log
I20240502 07:58:21.790239 6650 ts_tablet_manager.cc:1367] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240502 07:58:21.790365 6529 tablet_bootstrap.cc:492] T 7001a52697f1417bb84a0e3ed673ed0c P 91e82003a656442189a329b29846a841: No bootstrap required, opened a new log
I20240502 07:58:21.790439 6529 ts_tablet_manager.cc:1367] T 7001a52697f1417bb84a0e3ed673ed0c P 91e82003a656442189a329b29846a841: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:21.790637 6650 raft_consensus.cc:348] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } }
I20240502 07:58:21.790733 6650 raft_consensus.cc:374] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:21.790762 6650 raft_consensus.cc:729] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d58f6d1ff8cf472f8a9edf3700e64bfb, State: Initialized, Role: FOLLOWER
I20240502 07:58:21.790817 6529 raft_consensus.cc:348] T 7001a52697f1417bb84a0e3ed673ed0c P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } }
I20240502 07:58:21.790871 6650 consensus_queue.cc:260] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } }
I20240502 07:58:21.790933 6529 raft_consensus.cc:374] T 7001a52697f1417bb84a0e3ed673ed0c P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:21.790972 6529 raft_consensus.cc:729] T 7001a52697f1417bb84a0e3ed673ed0c P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91e82003a656442189a329b29846a841, State: Initialized, Role: FOLLOWER
I20240502 07:58:21.791122 6650 ts_tablet_manager.cc:1398] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:21.791116 6529 consensus_queue.cc:260] T 7001a52697f1417bb84a0e3ed673ed0c P 91e82003a656442189a329b29846a841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } }
I20240502 07:58:21.791354 6529 ts_tablet_manager.cc:1398] T 7001a52697f1417bb84a0e3ed673ed0c P 91e82003a656442189a329b29846a841: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:21.791514 6529 tablet_bootstrap.cc:492] T d41a2cb7b69e40d9a681384e5ad0a451 P 91e82003a656442189a329b29846a841: Bootstrap starting.
I20240502 07:58:21.791721 6542 tablet_bootstrap.cc:492] T 7001a52697f1417bb84a0e3ed673ed0c P 89bf5c0a6b2a4d949851089446f17b96: Bootstrap starting.
I20240502 07:58:21.791760 6651 tablet_bootstrap.cc:492] T d41a2cb7b69e40d9a681384e5ad0a451 P e020e7ffe8ce423a8bdefec667161217: No bootstrap required, opened a new log
I20240502 07:58:21.791824 6651 ts_tablet_manager.cc:1367] T d41a2cb7b69e40d9a681384e5ad0a451 P e020e7ffe8ce423a8bdefec667161217: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 07:58:21.792311 6529 tablet_bootstrap.cc:654] T d41a2cb7b69e40d9a681384e5ad0a451 P 91e82003a656442189a329b29846a841: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:21.792315 6651 raft_consensus.cc:348] T d41a2cb7b69e40d9a681384e5ad0a451 P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } }
I20240502 07:58:21.792414 6651 raft_consensus.cc:374] T d41a2cb7b69e40d9a681384e5ad0a451 P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:21.792444 6651 raft_consensus.cc:729] T d41a2cb7b69e40d9a681384e5ad0a451 P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e020e7ffe8ce423a8bdefec667161217, State: Initialized, Role: FOLLOWER
I20240502 07:58:21.792495 6542 tablet_bootstrap.cc:654] T 7001a52697f1417bb84a0e3ed673ed0c P 89bf5c0a6b2a4d949851089446f17b96: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:21.792584 6651 consensus_queue.cc:260] T d41a2cb7b69e40d9a681384e5ad0a451 P e020e7ffe8ce423a8bdefec667161217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } }
I20240502 07:58:21.792841 6651 ts_tablet_manager.cc:1398] T d41a2cb7b69e40d9a681384e5ad0a451 P e020e7ffe8ce423a8bdefec667161217: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:21.793665 6529 tablet_bootstrap.cc:492] T d41a2cb7b69e40d9a681384e5ad0a451 P 91e82003a656442189a329b29846a841: No bootstrap required, opened a new log
I20240502 07:58:21.793737 6529 ts_tablet_manager.cc:1367] T d41a2cb7b69e40d9a681384e5ad0a451 P 91e82003a656442189a329b29846a841: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:21.793943 6651 tablet_bootstrap.cc:492] T 7001a52697f1417bb84a0e3ed673ed0c P e020e7ffe8ce423a8bdefec667161217: Bootstrap starting.
I20240502 07:58:21.794138 6529 raft_consensus.cc:348] T d41a2cb7b69e40d9a681384e5ad0a451 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } }
I20240502 07:58:21.794236 6529 raft_consensus.cc:374] T d41a2cb7b69e40d9a681384e5ad0a451 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:21.794272 6529 raft_consensus.cc:729] T d41a2cb7b69e40d9a681384e5ad0a451 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91e82003a656442189a329b29846a841, State: Initialized, Role: FOLLOWER
I20240502 07:58:21.794375 6529 consensus_queue.cc:260] T d41a2cb7b69e40d9a681384e5ad0a451 P 91e82003a656442189a329b29846a841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } }
I20240502 07:58:21.794622 6529 ts_tablet_manager.cc:1398] T d41a2cb7b69e40d9a681384e5ad0a451 P 91e82003a656442189a329b29846a841: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:21.795022 6651 tablet_bootstrap.cc:654] T 7001a52697f1417bb84a0e3ed673ed0c P e020e7ffe8ce423a8bdefec667161217: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:21.795341 6542 tablet_bootstrap.cc:492] T 7001a52697f1417bb84a0e3ed673ed0c P 89bf5c0a6b2a4d949851089446f17b96: No bootstrap required, opened a new log
I20240502 07:58:21.795410 6542 ts_tablet_manager.cc:1367] T 7001a52697f1417bb84a0e3ed673ed0c P 89bf5c0a6b2a4d949851089446f17b96: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240502 07:58:21.795789 6542 raft_consensus.cc:348] T 7001a52697f1417bb84a0e3ed673ed0c P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } }
I20240502 07:58:21.795912 6542 raft_consensus.cc:374] T 7001a52697f1417bb84a0e3ed673ed0c P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:21.795951 6542 raft_consensus.cc:729] T 7001a52697f1417bb84a0e3ed673ed0c P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89bf5c0a6b2a4d949851089446f17b96, State: Initialized, Role: FOLLOWER
I20240502 07:58:21.796015 6651 tablet_bootstrap.cc:492] T 7001a52697f1417bb84a0e3ed673ed0c P e020e7ffe8ce423a8bdefec667161217: No bootstrap required, opened a new log
I20240502 07:58:21.796090 6651 ts_tablet_manager.cc:1367] T 7001a52697f1417bb84a0e3ed673ed0c P e020e7ffe8ce423a8bdefec667161217: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 07:58:21.796072 6542 consensus_queue.cc:260] T 7001a52697f1417bb84a0e3ed673ed0c P 89bf5c0a6b2a4d949851089446f17b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } }
I20240502 07:58:21.796308 6542 ts_tablet_manager.cc:1398] T 7001a52697f1417bb84a0e3ed673ed0c P 89bf5c0a6b2a4d949851089446f17b96: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:21.796471 6651 raft_consensus.cc:348] T 7001a52697f1417bb84a0e3ed673ed0c P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } }
I20240502 07:58:21.796555 6651 raft_consensus.cc:374] T 7001a52697f1417bb84a0e3ed673ed0c P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:21.796583 6651 raft_consensus.cc:729] T 7001a52697f1417bb84a0e3ed673ed0c P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e020e7ffe8ce423a8bdefec667161217, State: Initialized, Role: FOLLOWER
I20240502 07:58:21.796609 5872 tablet_service.cc:1467] Processing CreateTablet for tablet acf2ff61e8ff450ab7fca71fb4191dd6 (DEFAULT_TABLE table=rebalance_test_table_1 [id=6f5d44f6fbe544099b33ad94e35cee6e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:21.796692 6651 consensus_queue.cc:260] T 7001a52697f1417bb84a0e3ed673ed0c P e020e7ffe8ce423a8bdefec667161217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } }
I20240502 07:58:21.796809 5872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acf2ff61e8ff450ab7fca71fb4191dd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:21.796948 6651 ts_tablet_manager.cc:1398] T 7001a52697f1417bb84a0e3ed673ed0c P e020e7ffe8ce423a8bdefec667161217: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:21.797394 6651 tablet_bootstrap.cc:492] T acf2ff61e8ff450ab7fca71fb4191dd6 P e020e7ffe8ce423a8bdefec667161217: Bootstrap starting.
I20240502 07:58:21.798111 6542 tablet_bootstrap.cc:492] T acf2ff61e8ff450ab7fca71fb4191dd6 P 89bf5c0a6b2a4d949851089446f17b96: Bootstrap starting.
I20240502 07:58:21.798182 6651 tablet_bootstrap.cc:654] T acf2ff61e8ff450ab7fca71fb4191dd6 P e020e7ffe8ce423a8bdefec667161217: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:21.799146 6651 tablet_bootstrap.cc:492] T acf2ff61e8ff450ab7fca71fb4191dd6 P e020e7ffe8ce423a8bdefec667161217: No bootstrap required, opened a new log
I20240502 07:58:21.799185 6542 tablet_bootstrap.cc:654] T acf2ff61e8ff450ab7fca71fb4191dd6 P 89bf5c0a6b2a4d949851089446f17b96: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:21.799224 6651 ts_tablet_manager.cc:1367] T acf2ff61e8ff450ab7fca71fb4191dd6 P e020e7ffe8ce423a8bdefec667161217: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 07:58:21.799278 6655 tablet_bootstrap.cc:492] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688: Bootstrap starting.
I20240502 07:58:21.799561 6651 raft_consensus.cc:348] T acf2ff61e8ff450ab7fca71fb4191dd6 P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } }
I20240502 07:58:21.799641 6651 raft_consensus.cc:374] T acf2ff61e8ff450ab7fca71fb4191dd6 P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:21.799679 6651 raft_consensus.cc:729] T acf2ff61e8ff450ab7fca71fb4191dd6 P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e020e7ffe8ce423a8bdefec667161217, State: Initialized, Role: FOLLOWER
I20240502 07:58:21.799798 6651 consensus_queue.cc:260] T acf2ff61e8ff450ab7fca71fb4191dd6 P e020e7ffe8ce423a8bdefec667161217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } }
I20240502 07:58:21.800022 6655 tablet_bootstrap.cc:654] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:21.800025 6651 ts_tablet_manager.cc:1398] T acf2ff61e8ff450ab7fca71fb4191dd6 P e020e7ffe8ce423a8bdefec667161217: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:21.800154 6542 tablet_bootstrap.cc:492] T acf2ff61e8ff450ab7fca71fb4191dd6 P 89bf5c0a6b2a4d949851089446f17b96: No bootstrap required, opened a new log
I20240502 07:58:21.800231 6542 ts_tablet_manager.cc:1367] T acf2ff61e8ff450ab7fca71fb4191dd6 P 89bf5c0a6b2a4d949851089446f17b96: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 07:58:21.800530 6648 tablet_bootstrap.cc:492] T d41a2cb7b69e40d9a681384e5ad0a451 P d5bc7863cd294f77a932c4a40f98b79d: Bootstrap starting.
I20240502 07:58:21.800596 6542 raft_consensus.cc:348] T acf2ff61e8ff450ab7fca71fb4191dd6 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } }
I20240502 07:58:21.800678 6542 raft_consensus.cc:374] T acf2ff61e8ff450ab7fca71fb4191dd6 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:21.800704 6542 raft_consensus.cc:729] T acf2ff61e8ff450ab7fca71fb4191dd6 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89bf5c0a6b2a4d949851089446f17b96, State: Initialized, Role: FOLLOWER
I20240502 07:58:21.800807 6542 consensus_queue.cc:260] T acf2ff61e8ff450ab7fca71fb4191dd6 P 89bf5c0a6b2a4d949851089446f17b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } }
I20240502 07:58:21.801128 6542 ts_tablet_manager.cc:1398] T acf2ff61e8ff450ab7fca71fb4191dd6 P 89bf5c0a6b2a4d949851089446f17b96: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:21.801785 6655 tablet_bootstrap.cc:492] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688: No bootstrap required, opened a new log
I20240502 07:58:21.801852 6655 ts_tablet_manager.cc:1367] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:58:21.802196 6655 raft_consensus.cc:348] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } }
I20240502 07:58:21.802284 6655 raft_consensus.cc:374] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:21.802310 6655 raft_consensus.cc:729] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eda55a8566c447cfa749457825b70688, State: Initialized, Role: FOLLOWER
I20240502 07:58:21.802428 6655 consensus_queue.cc:260] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } }
I20240502 07:58:21.802516 6600 raft_consensus.cc:482] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:21.802600 6600 raft_consensus.cc:504] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } }
I20240502 07:58:21.802724 6655 ts_tablet_manager.cc:1398] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:21.803006 6648 tablet_bootstrap.cc:654] T d41a2cb7b69e40d9a681384e5ad0a451 P d5bc7863cd294f77a932c4a40f98b79d: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:21.803160 6600 leader_election.cc:288] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70:41047), d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68:44287), 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67:37141), e020e7ffe8ce423a8bdefec667161217 (127.3.139.69:39399)
I20240502 07:58:21.803426 6285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acf2ff61e8ff450ab7fca71fb4191dd6" candidate_uuid: "eda55a8566c447cfa749457825b70688" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" is_pre_election: true
I20240502 07:58:21.803521 6285 raft_consensus.cc:2454] T acf2ff61e8ff450ab7fca71fb4191dd6 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eda55a8566c447cfa749457825b70688 in term 0.
I20240502 07:58:21.803543 6023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acf2ff61e8ff450ab7fca71fb4191dd6" candidate_uuid: "eda55a8566c447cfa749457825b70688" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5bc7863cd294f77a932c4a40f98b79d" is_pre_election: true
I20240502 07:58:21.803550 6153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acf2ff61e8ff450ab7fca71fb4191dd6" candidate_uuid: "eda55a8566c447cfa749457825b70688" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e020e7ffe8ce423a8bdefec667161217" is_pre_election: true
I20240502 07:58:21.803650 6153 raft_consensus.cc:2454] T acf2ff61e8ff450ab7fca71fb4191dd6 P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eda55a8566c447cfa749457825b70688 in term 0.
I20240502 07:58:21.803985 5696 leader_election.cc:302] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: d58f6d1ff8cf472f8a9edf3700e64bfb, e020e7ffe8ce423a8bdefec667161217, eda55a8566c447cfa749457825b70688; no voters:
I20240502 07:58:21.804025 5891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acf2ff61e8ff450ab7fca71fb4191dd6" candidate_uuid: "eda55a8566c447cfa749457825b70688" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89bf5c0a6b2a4d949851089446f17b96" is_pre_election: true
W20240502 07:58:21.804080 5695 leader_election.cc:341] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68:44287): Illegal state: must be running to vote when last-logged opid is not known
I20240502 07:58:21.804134 5891 raft_consensus.cc:2454] T acf2ff61e8ff450ab7fca71fb4191dd6 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eda55a8566c447cfa749457825b70688 in term 0.
I20240502 07:58:21.804148 6600 raft_consensus.cc:2781] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:21.804193 6600 raft_consensus.cc:482] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:21.804229 6600 raft_consensus.cc:3037] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:21.805040 6600 raft_consensus.cc:504] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } }
I20240502 07:58:21.805492 6600 leader_election.cc:288] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688 [CANDIDATE]: Term 1 election: Requested vote from peers d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70:41047), d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68:44287), 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67:37141), e020e7ffe8ce423a8bdefec667161217 (127.3.139.69:39399)
I20240502 07:58:21.805626 6285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acf2ff61e8ff450ab7fca71fb4191dd6" candidate_uuid: "eda55a8566c447cfa749457825b70688" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb"
I20240502 07:58:21.805716 6285 raft_consensus.cc:3037] T acf2ff61e8ff450ab7fca71fb4191dd6 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:21.805735 6023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acf2ff61e8ff450ab7fca71fb4191dd6" candidate_uuid: "eda55a8566c447cfa749457825b70688" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5bc7863cd294f77a932c4a40f98b79d"
W20240502 07:58:21.806138 5695 leader_election.cc:341] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68:44287): Illegal state: must be running to vote when last-logged opid is not known
I20240502 07:58:21.806478 6285 raft_consensus.cc:2454] T acf2ff61e8ff450ab7fca71fb4191dd6 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eda55a8566c447cfa749457825b70688 in term 1.
I20240502 07:58:21.806643 5891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acf2ff61e8ff450ab7fca71fb4191dd6" candidate_uuid: "eda55a8566c447cfa749457825b70688" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89bf5c0a6b2a4d949851089446f17b96"
I20240502 07:58:21.806708 5891 raft_consensus.cc:3037] T acf2ff61e8ff450ab7fca71fb4191dd6 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:21.807319 5891 raft_consensus.cc:2454] T acf2ff61e8ff450ab7fca71fb4191dd6 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eda55a8566c447cfa749457825b70688 in term 1.
I20240502 07:58:21.807368 6153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acf2ff61e8ff450ab7fca71fb4191dd6" candidate_uuid: "eda55a8566c447cfa749457825b70688" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e020e7ffe8ce423a8bdefec667161217"
I20240502 07:58:21.807459 6153 raft_consensus.cc:3037] T acf2ff61e8ff450ab7fca71fb4191dd6 P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:21.807569 5695 leader_election.cc:302] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 5 voters: 3 yes votes; 1 no votes. yes voters: 89bf5c0a6b2a4d949851089446f17b96, d58f6d1ff8cf472f8a9edf3700e64bfb, eda55a8566c447cfa749457825b70688; no voters: d5bc7863cd294f77a932c4a40f98b79d
I20240502 07:58:21.807648 6648 tablet_bootstrap.cc:492] T d41a2cb7b69e40d9a681384e5ad0a451 P d5bc7863cd294f77a932c4a40f98b79d: No bootstrap required, opened a new log
I20240502 07:58:21.807723 6600 raft_consensus.cc:2781] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:21.807734 6648 ts_tablet_manager.cc:1367] T d41a2cb7b69e40d9a681384e5ad0a451 P d5bc7863cd294f77a932c4a40f98b79d: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240502 07:58:21.807793 6600 raft_consensus.cc:686] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688 [term 1 LEADER]: Becoming Leader. State: Replica: eda55a8566c447cfa749457825b70688, State: Running, Role: LEADER
I20240502 07:58:21.807926 6600 consensus_queue.cc:237] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } }
I20240502 07:58:21.808082 6648 raft_consensus.cc:348] T d41a2cb7b69e40d9a681384e5ad0a451 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } }
I20240502 07:58:21.808192 6648 raft_consensus.cc:374] T d41a2cb7b69e40d9a681384e5ad0a451 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:21.808233 6648 raft_consensus.cc:729] T d41a2cb7b69e40d9a681384e5ad0a451 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5bc7863cd294f77a932c4a40f98b79d, State: Initialized, Role: FOLLOWER
I20240502 07:58:21.808266 6153 raft_consensus.cc:2454] T acf2ff61e8ff450ab7fca71fb4191dd6 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eda55a8566c447cfa749457825b70688 in term 1.
I20240502 07:58:21.808354 6648 consensus_queue.cc:260] T d41a2cb7b69e40d9a681384e5ad0a451 P d5bc7863cd294f77a932c4a40f98b79d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } }
I20240502 07:58:21.808629 6648 ts_tablet_manager.cc:1398] T d41a2cb7b69e40d9a681384e5ad0a451 P d5bc7863cd294f77a932c4a40f98b79d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:21.809518 5484 catalog_manager.cc:5484] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688 reported cstate change: term changed from 0 to 1, leader changed from <none> to eda55a8566c447cfa749457825b70688 (127.3.139.66). New cstate: current_term: 1 leader_uuid: "eda55a8566c447cfa749457825b70688" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:21.809738 6648 tablet_bootstrap.cc:492] T acf2ff61e8ff450ab7fca71fb4191dd6 P d5bc7863cd294f77a932c4a40f98b79d: Bootstrap starting.
I20240502 07:58:21.810503 6648 tablet_bootstrap.cc:654] T acf2ff61e8ff450ab7fca71fb4191dd6 P d5bc7863cd294f77a932c4a40f98b79d: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:21.811599 6648 tablet_bootstrap.cc:492] T acf2ff61e8ff450ab7fca71fb4191dd6 P d5bc7863cd294f77a932c4a40f98b79d: No bootstrap required, opened a new log
I20240502 07:58:21.811666 6648 ts_tablet_manager.cc:1367] T acf2ff61e8ff450ab7fca71fb4191dd6 P d5bc7863cd294f77a932c4a40f98b79d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:58:21.811973 6648 raft_consensus.cc:348] T acf2ff61e8ff450ab7fca71fb4191dd6 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } }
I20240502 07:58:21.812050 6648 raft_consensus.cc:374] T acf2ff61e8ff450ab7fca71fb4191dd6 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:21.812069 6648 raft_consensus.cc:729] T acf2ff61e8ff450ab7fca71fb4191dd6 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5bc7863cd294f77a932c4a40f98b79d, State: Initialized, Role: FOLLOWER
I20240502 07:58:21.812163 6648 consensus_queue.cc:260] T acf2ff61e8ff450ab7fca71fb4191dd6 P d5bc7863cd294f77a932c4a40f98b79d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } }
I20240502 07:58:21.812381 6648 ts_tablet_manager.cc:1398] T acf2ff61e8ff450ab7fca71fb4191dd6 P d5bc7863cd294f77a932c4a40f98b79d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:21.812569 6648 tablet_bootstrap.cc:492] T 7001a52697f1417bb84a0e3ed673ed0c P d5bc7863cd294f77a932c4a40f98b79d: Bootstrap starting.
I20240502 07:58:21.813186 6548 raft_consensus.cc:482] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:21.813279 6548 raft_consensus.cc:504] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } }
I20240502 07:58:21.813764 6648 tablet_bootstrap.cc:654] T 7001a52697f1417bb84a0e3ed673ed0c P d5bc7863cd294f77a932c4a40f98b79d: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:21.814643 6548 leader_election.cc:288] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e020e7ffe8ce423a8bdefec667161217 (127.3.139.69:39399), 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67:37141), 91e82003a656442189a329b29846a841 (127.3.139.65:43429), d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68:44287)
I20240502 07:58:21.814750 6648 tablet_bootstrap.cc:492] T 7001a52697f1417bb84a0e3ed673ed0c P d5bc7863cd294f77a932c4a40f98b79d: No bootstrap required, opened a new log
I20240502 07:58:21.814819 6648 ts_tablet_manager.cc:1367] T 7001a52697f1417bb84a0e3ed673ed0c P d5bc7863cd294f77a932c4a40f98b79d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:21.815223 6648 raft_consensus.cc:348] T 7001a52697f1417bb84a0e3ed673ed0c P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } }
I20240502 07:58:21.815312 6648 raft_consensus.cc:374] T 7001a52697f1417bb84a0e3ed673ed0c P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:21.815341 6648 raft_consensus.cc:729] T 7001a52697f1417bb84a0e3ed673ed0c P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5bc7863cd294f77a932c4a40f98b79d, State: Initialized, Role: FOLLOWER
I20240502 07:58:21.815448 6648 consensus_queue.cc:260] T 7001a52697f1417bb84a0e3ed673ed0c P d5bc7863cd294f77a932c4a40f98b79d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } }
I20240502 07:58:21.815685 6648 ts_tablet_manager.cc:1398] T 7001a52697f1417bb84a0e3ed673ed0c P d5bc7863cd294f77a932c4a40f98b79d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:21.819559 5891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7001a52697f1417bb84a0e3ed673ed0c" candidate_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89bf5c0a6b2a4d949851089446f17b96" is_pre_election: true
I20240502 07:58:21.819659 5891 raft_consensus.cc:2454] T 7001a52697f1417bb84a0e3ed673ed0c P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d58f6d1ff8cf472f8a9edf3700e64bfb in term 0.
I20240502 07:58:21.819737 6153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7001a52697f1417bb84a0e3ed673ed0c" candidate_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e020e7ffe8ce423a8bdefec667161217" is_pre_election: true
I20240502 07:58:21.819841 6153 raft_consensus.cc:2454] T 7001a52697f1417bb84a0e3ed673ed0c P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d58f6d1ff8cf472f8a9edf3700e64bfb in term 0.
I20240502 07:58:21.820820 6220 leader_election.cc:302] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 89bf5c0a6b2a4d949851089446f17b96, d58f6d1ff8cf472f8a9edf3700e64bfb, e020e7ffe8ce423a8bdefec667161217; no voters:
I20240502 07:58:21.821035 6548 raft_consensus.cc:2781] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:21.821089 6548 raft_consensus.cc:482] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:21.821133 6548 raft_consensus.cc:3037] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:21.821342 6023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7001a52697f1417bb84a0e3ed673ed0c" candidate_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5bc7863cd294f77a932c4a40f98b79d" is_pre_election: true
I20240502 07:58:21.821475 6023 raft_consensus.cc:2454] T 7001a52697f1417bb84a0e3ed673ed0c P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d58f6d1ff8cf472f8a9edf3700e64bfb in term 0.
I20240502 07:58:21.821980 6548 raft_consensus.cc:504] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } }
I20240502 07:58:21.822077 6674 raft_consensus.cc:482] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:21.822077 5629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7001a52697f1417bb84a0e3ed673ed0c" candidate_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e82003a656442189a329b29846a841" is_pre_election: true
I20240502 07:58:21.822176 5629 raft_consensus.cc:2454] T 7001a52697f1417bb84a0e3ed673ed0c P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d58f6d1ff8cf472f8a9edf3700e64bfb in term 0.
I20240502 07:58:21.822154 6674 raft_consensus.cc:504] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } }
I20240502 07:58:21.822389 6548 leader_election.cc:288] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb [CANDIDATE]: Term 1 election: Requested vote from peers e020e7ffe8ce423a8bdefec667161217 (127.3.139.69:39399), 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67:37141), 91e82003a656442189a329b29846a841 (127.3.139.65:43429), d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68:44287)
I20240502 07:58:21.822607 6153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7001a52697f1417bb84a0e3ed673ed0c" candidate_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e020e7ffe8ce423a8bdefec667161217"
I20240502 07:58:21.822710 6153 raft_consensus.cc:3037] T 7001a52697f1417bb84a0e3ed673ed0c P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:21.822810 5891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7001a52697f1417bb84a0e3ed673ed0c" candidate_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89bf5c0a6b2a4d949851089446f17b96"
I20240502 07:58:21.822866 6154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d41a2cb7b69e40d9a681384e5ad0a451" candidate_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e020e7ffe8ce423a8bdefec667161217" is_pre_election: true
I20240502 07:58:21.822908 5891 raft_consensus.cc:3037] T 7001a52697f1417bb84a0e3ed673ed0c P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:21.822952 6154 raft_consensus.cc:2454] T d41a2cb7b69e40d9a681384e5ad0a451 P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d58f6d1ff8cf472f8a9edf3700e64bfb in term 0.
I20240502 07:58:21.823336 5890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d41a2cb7b69e40d9a681384e5ad0a451" candidate_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89bf5c0a6b2a4d949851089446f17b96" is_pre_election: true
I20240502 07:58:21.823444 5890 raft_consensus.cc:2454] T d41a2cb7b69e40d9a681384e5ad0a451 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d58f6d1ff8cf472f8a9edf3700e64bfb in term 0.
I20240502 07:58:21.823525 6153 raft_consensus.cc:2454] T 7001a52697f1417bb84a0e3ed673ed0c P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d58f6d1ff8cf472f8a9edf3700e64bfb in term 1.
I20240502 07:58:21.823642 5891 raft_consensus.cc:2454] T 7001a52697f1417bb84a0e3ed673ed0c P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d58f6d1ff8cf472f8a9edf3700e64bfb in term 1.
I20240502 07:58:21.823623 5629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7001a52697f1417bb84a0e3ed673ed0c" candidate_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e82003a656442189a329b29846a841"
I20240502 07:58:21.823669 5628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d41a2cb7b69e40d9a681384e5ad0a451" candidate_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e82003a656442189a329b29846a841" is_pre_election: true
I20240502 07:58:21.823711 5629 raft_consensus.cc:3037] T 7001a52697f1417bb84a0e3ed673ed0c P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:21.823765 5628 raft_consensus.cc:2454] T d41a2cb7b69e40d9a681384e5ad0a451 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d58f6d1ff8cf472f8a9edf3700e64bfb in term 0.
I20240502 07:58:21.824064 6219 leader_election.cc:302] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 91e82003a656442189a329b29846a841, d58f6d1ff8cf472f8a9edf3700e64bfb, e020e7ffe8ce423a8bdefec667161217; no voters:
I20240502 07:58:21.824174 6674 leader_election.cc:288] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67:37141), d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68:44287), e020e7ffe8ce423a8bdefec667161217 (127.3.139.69:39399), 91e82003a656442189a329b29846a841 (127.3.139.65:43429)
I20240502 07:58:21.824223 6548 raft_consensus.cc:2781] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:21.824271 6548 raft_consensus.cc:482] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:21.824304 6548 raft_consensus.cc:3037] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:21.824445 5629 raft_consensus.cc:2454] T 7001a52697f1417bb84a0e3ed673ed0c P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d58f6d1ff8cf472f8a9edf3700e64bfb in term 1.
I20240502 07:58:21.824713 6219 leader_election.cc:302] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 91e82003a656442189a329b29846a841, d58f6d1ff8cf472f8a9edf3700e64bfb, e020e7ffe8ce423a8bdefec667161217; no voters:
I20240502 07:58:21.824894 6674 raft_consensus.cc:2781] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:21.825014 6548 raft_consensus.cc:504] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } }
I20240502 07:58:21.825059 6022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d41a2cb7b69e40d9a681384e5ad0a451" candidate_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5bc7863cd294f77a932c4a40f98b79d" is_pre_election: true
I20240502 07:58:21.825074 6023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7001a52697f1417bb84a0e3ed673ed0c" candidate_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5bc7863cd294f77a932c4a40f98b79d"
I20240502 07:58:21.825050 6674 raft_consensus.cc:686] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 LEADER]: Becoming Leader. State: Replica: d58f6d1ff8cf472f8a9edf3700e64bfb, State: Running, Role: LEADER
I20240502 07:58:21.825157 6023 raft_consensus.cc:3037] T 7001a52697f1417bb84a0e3ed673ed0c P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:21.825156 6022 raft_consensus.cc:2454] T d41a2cb7b69e40d9a681384e5ad0a451 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d58f6d1ff8cf472f8a9edf3700e64bfb in term 0.
I20240502 07:58:21.825366 6674 consensus_queue.cc:237] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } }
I20240502 07:58:21.825457 6548 leader_election.cc:288] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [CANDIDATE]: Term 1 election: Requested vote from peers 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67:37141), d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68:44287), e020e7ffe8ce423a8bdefec667161217 (127.3.139.69:39399), 91e82003a656442189a329b29846a841 (127.3.139.65:43429)
I20240502 07:58:21.825822 6153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d41a2cb7b69e40d9a681384e5ad0a451" candidate_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e020e7ffe8ce423a8bdefec667161217"
I20240502 07:58:21.825920 6153 raft_consensus.cc:3037] T d41a2cb7b69e40d9a681384e5ad0a451 P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:21.826030 6023 raft_consensus.cc:2454] T 7001a52697f1417bb84a0e3ed673ed0c P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d58f6d1ff8cf472f8a9edf3700e64bfb in term 1.
I20240502 07:58:21.826555 6153 raft_consensus.cc:2454] T d41a2cb7b69e40d9a681384e5ad0a451 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d58f6d1ff8cf472f8a9edf3700e64bfb in term 1.
I20240502 07:58:21.827159 5629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d41a2cb7b69e40d9a681384e5ad0a451" candidate_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e82003a656442189a329b29846a841"
I20240502 07:58:21.827227 5891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d41a2cb7b69e40d9a681384e5ad0a451" candidate_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89bf5c0a6b2a4d949851089446f17b96"
I20240502 07:58:21.827257 5629 raft_consensus.cc:3037] T d41a2cb7b69e40d9a681384e5ad0a451 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:21.827308 5891 raft_consensus.cc:3037] T d41a2cb7b69e40d9a681384e5ad0a451 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:21.827442 5487 catalog_manager.cc:5484] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb reported cstate change: term changed from 0 to 1, leader changed from <none> to d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70). New cstate: current_term: 1 leader_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:21.827821 5891 raft_consensus.cc:2454] T d41a2cb7b69e40d9a681384e5ad0a451 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d58f6d1ff8cf472f8a9edf3700e64bfb in term 1.
I20240502 07:58:21.828006 5629 raft_consensus.cc:2454] T d41a2cb7b69e40d9a681384e5ad0a451 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d58f6d1ff8cf472f8a9edf3700e64bfb in term 1.
I20240502 07:58:21.828074 6219 leader_election.cc:302] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 89bf5c0a6b2a4d949851089446f17b96, d58f6d1ff8cf472f8a9edf3700e64bfb, e020e7ffe8ce423a8bdefec667161217; no voters:
I20240502 07:58:21.828244 6675 raft_consensus.cc:2781] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:21.828330 6675 raft_consensus.cc:686] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 LEADER]: Becoming Leader. State: Replica: d58f6d1ff8cf472f8a9edf3700e64bfb, State: Running, Role: LEADER
I20240502 07:58:21.828454 6675 consensus_queue.cc:237] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } }
I20240502 07:58:21.828785 6023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d41a2cb7b69e40d9a681384e5ad0a451" candidate_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5bc7863cd294f77a932c4a40f98b79d"
I20240502 07:58:21.828879 6023 raft_consensus.cc:3037] T d41a2cb7b69e40d9a681384e5ad0a451 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:21.829600 6023 raft_consensus.cc:2454] T d41a2cb7b69e40d9a681384e5ad0a451 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d58f6d1ff8cf472f8a9edf3700e64bfb in term 1.
I20240502 07:58:21.830201 5487 catalog_manager.cc:5484] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb reported cstate change: term changed from 0 to 1, leader changed from <none> to d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70). New cstate: current_term: 1 leader_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } health_report { overall_health: HEALTHY } } }
W20240502 07:58:21.942881 6678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:21.943106 6678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:21.946059 6678 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_1'
I20240502 07:58:21.966774 6560 raft_consensus.cc:482] T acf2ff61e8ff450ab7fca71fb4191dd6 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:21.966866 6560 raft_consensus.cc:504] T acf2ff61e8ff450ab7fca71fb4191dd6 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } }
I20240502 07:58:21.967370 6560 leader_election.cc:288] T acf2ff61e8ff450ab7fca71fb4191dd6 P d5bc7863cd294f77a932c4a40f98b79d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70:41047), 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67:37141), eda55a8566c447cfa749457825b70688 (127.3.139.66:39925), e020e7ffe8ce423a8bdefec667161217 (127.3.139.69:39399)
I20240502 07:58:21.967633 6285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acf2ff61e8ff450ab7fca71fb4191dd6" candidate_uuid: "d5bc7863cd294f77a932c4a40f98b79d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" is_pre_election: true
I20240502 07:58:21.967726 5891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acf2ff61e8ff450ab7fca71fb4191dd6" candidate_uuid: "d5bc7863cd294f77a932c4a40f98b79d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89bf5c0a6b2a4d949851089446f17b96" is_pre_election: true
I20240502 07:58:21.967788 6285 raft_consensus.cc:2379] T acf2ff61e8ff450ab7fca71fb4191dd6 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d5bc7863cd294f77a932c4a40f98b79d in current term 1: Already voted for candidate eda55a8566c447cfa749457825b70688 in this term.
I20240502 07:58:21.967912 5891 raft_consensus.cc:2379] T acf2ff61e8ff450ab7fca71fb4191dd6 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d5bc7863cd294f77a932c4a40f98b79d in current term 1: Already voted for candidate eda55a8566c447cfa749457825b70688 in this term.
I20240502 07:58:21.968004 6153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acf2ff61e8ff450ab7fca71fb4191dd6" candidate_uuid: "d5bc7863cd294f77a932c4a40f98b79d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e020e7ffe8ce423a8bdefec667161217" is_pre_election: true
I20240502 07:58:21.968137 6153 raft_consensus.cc:2379] T acf2ff61e8ff450ab7fca71fb4191dd6 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d5bc7863cd294f77a932c4a40f98b79d in current term 1: Already voted for candidate eda55a8566c447cfa749457825b70688 in this term.
I20240502 07:58:21.968173 5759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acf2ff61e8ff450ab7fca71fb4191dd6" candidate_uuid: "d5bc7863cd294f77a932c4a40f98b79d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eda55a8566c447cfa749457825b70688" is_pre_election: true
I20240502 07:58:21.968570 5957 leader_election.cc:302] T acf2ff61e8ff450ab7fca71fb4191dd6 P d5bc7863cd294f77a932c4a40f98b79d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 5 voters: 1 yes votes; 3 no votes. yes voters: d5bc7863cd294f77a932c4a40f98b79d; no voters: d58f6d1ff8cf472f8a9edf3700e64bfb, e020e7ffe8ce423a8bdefec667161217, eda55a8566c447cfa749457825b70688
I20240502 07:58:21.968734 6560 raft_consensus.cc:3037] T acf2ff61e8ff450ab7fca71fb4191dd6 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:21.969661 6560 raft_consensus.cc:2726] T acf2ff61e8ff450ab7fca71fb4191dd6 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 07:58:22.019405 6153 raft_consensus.cc:1261] T acf2ff61e8ff450ab7fca71fb4191dd6 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer eda55a8566c447cfa749457825b70688: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:22.019649 5891 raft_consensus.cc:1261] T acf2ff61e8ff450ab7fca71fb4191dd6 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer eda55a8566c447cfa749457825b70688: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:22.019729 6285 raft_consensus.cc:1261] T acf2ff61e8ff450ab7fca71fb4191dd6 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Refusing update from remote peer eda55a8566c447cfa749457825b70688: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:22.019964 6023 raft_consensus.cc:1261] T acf2ff61e8ff450ab7fca71fb4191dd6 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Refusing update from remote peer eda55a8566c447cfa749457825b70688: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:22.020143 6540 consensus_queue.cc:1035] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:22.020274 6545 consensus_queue.cc:1035] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:22.020905 6545 consensus_queue.cc:1035] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:22.021181 6545 consensus_queue.cc:1035] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:22.023257 5890 raft_consensus.cc:1261] T d41a2cb7b69e40d9a681384e5ad0a451 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:22.023718 6674 consensus_queue.cc:1035] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:22.023900 5629 raft_consensus.cc:1261] T 7001a52697f1417bb84a0e3ed673ed0c P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:22.024184 5628 raft_consensus.cc:1261] T d41a2cb7b69e40d9a681384e5ad0a451 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:22.024405 6154 raft_consensus.cc:1261] T d41a2cb7b69e40d9a681384e5ad0a451 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:22.025166 6022 raft_consensus.cc:1261] T d41a2cb7b69e40d9a681384e5ad0a451 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:22.026019 6674 consensus_queue.cc:1035] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:22.031077 6023 raft_consensus.cc:1261] T 7001a52697f1417bb84a0e3ed673ed0c P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:22.031250 6675 consensus_queue.cc:1035] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:22.031523 6675 consensus_queue.cc:1035] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:22.031584 6548 consensus_queue.cc:1035] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:22.034919 6548 consensus_queue.cc:1035] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:22.056622 5892 raft_consensus.cc:1261] T 7001a52697f1417bb84a0e3ed673ed0c P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:22.056938 6153 raft_consensus.cc:1261] T 7001a52697f1417bb84a0e3ed673ed0c P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:22.057516 6675 consensus_queue.cc:1035] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:22.058112 6674 consensus_queue.cc:1035] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 135.538 ms
time per row: 0.0677692 ms
I20240502 07:58:22.122186 5485 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40652:
name: "rebalance_test_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 5
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:58:22.127022 5740 tablet_service.cc:1467] Processing CreateTablet for tablet 18a448dbe2af493abefbc728a8f85186 (DEFAULT_TABLE table=rebalance_test_table_2 [id=28039a31ea894d23af67c1f1da387f21]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:22.127223 5872 tablet_service.cc:1467] Processing CreateTablet for tablet 18a448dbe2af493abefbc728a8f85186 (DEFAULT_TABLE table=rebalance_test_table_2 [id=28039a31ea894d23af67c1f1da387f21]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:22.127344 5740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18a448dbe2af493abefbc728a8f85186. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:22.127489 5872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18a448dbe2af493abefbc728a8f85186. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:22.128823 6133 tablet_service.cc:1467] Processing CreateTablet for tablet 18a448dbe2af493abefbc728a8f85186 (DEFAULT_TABLE table=rebalance_test_table_2 [id=28039a31ea894d23af67c1f1da387f21]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:22.128971 6263 tablet_service.cc:1467] Processing CreateTablet for tablet 5c8c7437c85943ba9113595c17bf6136 (DEFAULT_TABLE table=rebalance_test_table_2 [id=28039a31ea894d23af67c1f1da387f21]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:22.129068 6133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18a448dbe2af493abefbc728a8f85186. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:22.129074 6655 tablet_bootstrap.cc:492] T 18a448dbe2af493abefbc728a8f85186 P eda55a8566c447cfa749457825b70688: Bootstrap starting.
I20240502 07:58:22.129205 6263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c8c7437c85943ba9113595c17bf6136. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:22.129652 5610 tablet_service.cc:1467] Processing CreateTablet for tablet 18a448dbe2af493abefbc728a8f85186 (DEFAULT_TABLE table=rebalance_test_table_2 [id=28039a31ea894d23af67c1f1da387f21]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:22.129876 5610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18a448dbe2af493abefbc728a8f85186. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:22.129889 6655 tablet_bootstrap.cc:654] T 18a448dbe2af493abefbc728a8f85186 P eda55a8566c447cfa749457825b70688: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:22.130833 6651 tablet_bootstrap.cc:492] T 18a448dbe2af493abefbc728a8f85186 P e020e7ffe8ce423a8bdefec667161217: Bootstrap starting.
I20240502 07:58:22.131013 6132 tablet_service.cc:1467] Processing CreateTablet for tablet 635b0a1a0fd64eeeac70025014c9ab81 (DEFAULT_TABLE table=rebalance_test_table_2 [id=28039a31ea894d23af67c1f1da387f21]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:22.131242 6132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 635b0a1a0fd64eeeac70025014c9ab81. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:22.131269 6265 tablet_service.cc:1467] Processing CreateTablet for tablet 635b0a1a0fd64eeeac70025014c9ab81 (DEFAULT_TABLE table=rebalance_test_table_2 [id=28039a31ea894d23af67c1f1da387f21]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:22.131476 6265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 635b0a1a0fd64eeeac70025014c9ab81. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:22.131637 6651 tablet_bootstrap.cc:654] T 18a448dbe2af493abefbc728a8f85186 P e020e7ffe8ce423a8bdefec667161217: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:22.132161 6003 tablet_service.cc:1467] Processing CreateTablet for tablet 18a448dbe2af493abefbc728a8f85186 (DEFAULT_TABLE table=rebalance_test_table_2 [id=28039a31ea894d23af67c1f1da387f21]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:22.132402 6003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18a448dbe2af493abefbc728a8f85186. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:22.133201 6650 tablet_bootstrap.cc:492] T 5c8c7437c85943ba9113595c17bf6136 P d58f6d1ff8cf472f8a9edf3700e64bfb: Bootstrap starting.
I20240502 07:58:22.133945 6650 tablet_bootstrap.cc:654] T 5c8c7437c85943ba9113595c17bf6136 P d58f6d1ff8cf472f8a9edf3700e64bfb: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:22.134284 6648 tablet_bootstrap.cc:492] T 18a448dbe2af493abefbc728a8f85186 P d5bc7863cd294f77a932c4a40f98b79d: Bootstrap starting.
I20240502 07:58:22.134369 6542 tablet_bootstrap.cc:492] T 18a448dbe2af493abefbc728a8f85186 P 89bf5c0a6b2a4d949851089446f17b96: Bootstrap starting.
I20240502 07:58:22.135037 6648 tablet_bootstrap.cc:654] T 18a448dbe2af493abefbc728a8f85186 P d5bc7863cd294f77a932c4a40f98b79d: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:22.135141 6542 tablet_bootstrap.cc:654] T 18a448dbe2af493abefbc728a8f85186 P 89bf5c0a6b2a4d949851089446f17b96: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:22.135147 5608 tablet_service.cc:1467] Processing CreateTablet for tablet 635b0a1a0fd64eeeac70025014c9ab81 (DEFAULT_TABLE table=rebalance_test_table_2 [id=28039a31ea894d23af67c1f1da387f21]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:22.135360 5608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 635b0a1a0fd64eeeac70025014c9ab81. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:22.135376 6003 tablet_service.cc:1467] Processing CreateTablet for tablet 5c8c7437c85943ba9113595c17bf6136 (DEFAULT_TABLE table=rebalance_test_table_2 [id=28039a31ea894d23af67c1f1da387f21]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:22.135593 6003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c8c7437c85943ba9113595c17bf6136. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:22.135653 6529 tablet_bootstrap.cc:492] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841: Bootstrap starting.
I20240502 07:58:22.135936 5739 tablet_service.cc:1467] Processing CreateTablet for tablet 635b0a1a0fd64eeeac70025014c9ab81 (DEFAULT_TABLE table=rebalance_test_table_2 [id=28039a31ea894d23af67c1f1da387f21]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:22.136176 5739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 635b0a1a0fd64eeeac70025014c9ab81. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:22.136466 6529 tablet_bootstrap.cc:654] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:22.137388 5870 tablet_service.cc:1467] Processing CreateTablet for tablet 635b0a1a0fd64eeeac70025014c9ab81 (DEFAULT_TABLE table=rebalance_test_table_2 [id=28039a31ea894d23af67c1f1da387f21]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:22.137598 5870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 635b0a1a0fd64eeeac70025014c9ab81. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:22.137812 6650 tablet_bootstrap.cc:492] T 5c8c7437c85943ba9113595c17bf6136 P d58f6d1ff8cf472f8a9edf3700e64bfb: No bootstrap required, opened a new log
I20240502 07:58:22.137883 6650 ts_tablet_manager.cc:1367] T 5c8c7437c85943ba9113595c17bf6136 P d58f6d1ff8cf472f8a9edf3700e64bfb: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 07:58:22.138273 5610 tablet_service.cc:1467] Processing CreateTablet for tablet 5c8c7437c85943ba9113595c17bf6136 (DEFAULT_TABLE table=rebalance_test_table_2 [id=28039a31ea894d23af67c1f1da387f21]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:22.138273 6650 raft_consensus.cc:348] T 5c8c7437c85943ba9113595c17bf6136 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } }
I20240502 07:58:22.138371 6650 raft_consensus.cc:374] T 5c8c7437c85943ba9113595c17bf6136 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:22.138413 6650 raft_consensus.cc:729] T 5c8c7437c85943ba9113595c17bf6136 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d58f6d1ff8cf472f8a9edf3700e64bfb, State: Initialized, Role: FOLLOWER
I20240502 07:58:22.138489 5610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c8c7437c85943ba9113595c17bf6136. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:22.138531 6650 consensus_queue.cc:260] T 5c8c7437c85943ba9113595c17bf6136 P d58f6d1ff8cf472f8a9edf3700e64bfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } }
I20240502 07:58:22.138803 6650 ts_tablet_manager.cc:1398] T 5c8c7437c85943ba9113595c17bf6136 P d58f6d1ff8cf472f8a9edf3700e64bfb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:22.138767 5740 tablet_service.cc:1467] Processing CreateTablet for tablet 5c8c7437c85943ba9113595c17bf6136 (DEFAULT_TABLE table=rebalance_test_table_2 [id=28039a31ea894d23af67c1f1da387f21]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:22.138948 6650 tablet_bootstrap.cc:492] T 635b0a1a0fd64eeeac70025014c9ab81 P d58f6d1ff8cf472f8a9edf3700e64bfb: Bootstrap starting.
I20240502 07:58:22.138952 5740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c8c7437c85943ba9113595c17bf6136. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:22.139611 5872 tablet_service.cc:1467] Processing CreateTablet for tablet 5c8c7437c85943ba9113595c17bf6136 (DEFAULT_TABLE table=rebalance_test_table_2 [id=28039a31ea894d23af67c1f1da387f21]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:22.139837 5872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c8c7437c85943ba9113595c17bf6136. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:22.140235 6650 tablet_bootstrap.cc:654] T 635b0a1a0fd64eeeac70025014c9ab81 P d58f6d1ff8cf472f8a9edf3700e64bfb: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:22.141201 6650 tablet_bootstrap.cc:492] T 635b0a1a0fd64eeeac70025014c9ab81 P d58f6d1ff8cf472f8a9edf3700e64bfb: No bootstrap required, opened a new log
I20240502 07:58:22.141280 6650 ts_tablet_manager.cc:1367] T 635b0a1a0fd64eeeac70025014c9ab81 P d58f6d1ff8cf472f8a9edf3700e64bfb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:22.141394 6648 tablet_bootstrap.cc:492] T 18a448dbe2af493abefbc728a8f85186 P d5bc7863cd294f77a932c4a40f98b79d: No bootstrap required, opened a new log
I20240502 07:58:22.141458 6648 ts_tablet_manager.cc:1367] T 18a448dbe2af493abefbc728a8f85186 P d5bc7863cd294f77a932c4a40f98b79d: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20240502 07:58:22.141688 6650 raft_consensus.cc:348] T 635b0a1a0fd64eeeac70025014c9ab81 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } }
I20240502 07:58:22.141781 6650 raft_consensus.cc:374] T 635b0a1a0fd64eeeac70025014c9ab81 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:22.141810 6650 raft_consensus.cc:729] T 635b0a1a0fd64eeeac70025014c9ab81 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d58f6d1ff8cf472f8a9edf3700e64bfb, State: Initialized, Role: FOLLOWER
I20240502 07:58:22.141798 6648 raft_consensus.cc:348] T 18a448dbe2af493abefbc728a8f85186 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } }
I20240502 07:58:22.141881 6648 raft_consensus.cc:374] T 18a448dbe2af493abefbc728a8f85186 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:22.141906 6648 raft_consensus.cc:729] T 18a448dbe2af493abefbc728a8f85186 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5bc7863cd294f77a932c4a40f98b79d, State: Initialized, Role: FOLLOWER
I20240502 07:58:22.141920 6650 consensus_queue.cc:260] T 635b0a1a0fd64eeeac70025014c9ab81 P d58f6d1ff8cf472f8a9edf3700e64bfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } }
I20240502 07:58:22.142007 6648 consensus_queue.cc:260] T 18a448dbe2af493abefbc728a8f85186 P d5bc7863cd294f77a932c4a40f98b79d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } }
I20240502 07:58:22.142164 6650 ts_tablet_manager.cc:1398] T 635b0a1a0fd64eeeac70025014c9ab81 P d58f6d1ff8cf472f8a9edf3700e64bfb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:22.142225 6648 ts_tablet_manager.cc:1398] T 18a448dbe2af493abefbc728a8f85186 P d5bc7863cd294f77a932c4a40f98b79d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:22.142342 6648 tablet_bootstrap.cc:492] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d: Bootstrap starting.
I20240502 07:58:22.143030 6648 tablet_bootstrap.cc:654] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:22.143018 6655 tablet_bootstrap.cc:492] T 18a448dbe2af493abefbc728a8f85186 P eda55a8566c447cfa749457825b70688: No bootstrap required, opened a new log
I20240502 07:58:22.143133 6655 ts_tablet_manager.cc:1367] T 18a448dbe2af493abefbc728a8f85186 P eda55a8566c447cfa749457825b70688: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240502 07:58:22.143561 6655 raft_consensus.cc:348] T 18a448dbe2af493abefbc728a8f85186 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } }
I20240502 07:58:22.143666 6655 raft_consensus.cc:374] T 18a448dbe2af493abefbc728a8f85186 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:22.143697 6655 raft_consensus.cc:729] T 18a448dbe2af493abefbc728a8f85186 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eda55a8566c447cfa749457825b70688, State: Initialized, Role: FOLLOWER
I20240502 07:58:22.143796 6655 consensus_queue.cc:260] T 18a448dbe2af493abefbc728a8f85186 P eda55a8566c447cfa749457825b70688 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } }
I20240502 07:58:22.144064 6655 ts_tablet_manager.cc:1398] T 18a448dbe2af493abefbc728a8f85186 P eda55a8566c447cfa749457825b70688: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:22.144248 6655 tablet_bootstrap.cc:492] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688: Bootstrap starting.
I20240502 07:58:22.145201 6655 tablet_bootstrap.cc:654] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:22.146211 6655 tablet_bootstrap.cc:492] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688: No bootstrap required, opened a new log
I20240502 07:58:22.146289 6655 ts_tablet_manager.cc:1367] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:22.146692 6655 raft_consensus.cc:348] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } }
I20240502 07:58:22.146790 6655 raft_consensus.cc:374] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:22.146821 6655 raft_consensus.cc:729] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eda55a8566c447cfa749457825b70688, State: Initialized, Role: FOLLOWER
I20240502 07:58:22.146837 6648 tablet_bootstrap.cc:492] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d: No bootstrap required, opened a new log
I20240502 07:58:22.146843 6542 tablet_bootstrap.cc:492] T 18a448dbe2af493abefbc728a8f85186 P 89bf5c0a6b2a4d949851089446f17b96: No bootstrap required, opened a new log
I20240502 07:58:22.146906 6648 ts_tablet_manager.cc:1367] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20240502 07:58:22.146921 6542 ts_tablet_manager.cc:1367] T 18a448dbe2af493abefbc728a8f85186 P 89bf5c0a6b2a4d949851089446f17b96: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.001s
I20240502 07:58:22.146922 6655 consensus_queue.cc:260] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } }
I20240502 07:58:22.147213 6655 ts_tablet_manager.cc:1398] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:22.147306 6648 raft_consensus.cc:348] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } }
I20240502 07:58:22.147331 6542 raft_consensus.cc:348] T 18a448dbe2af493abefbc728a8f85186 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } }
I20240502 07:58:22.147397 6655 tablet_bootstrap.cc:492] T 5c8c7437c85943ba9113595c17bf6136 P eda55a8566c447cfa749457825b70688: Bootstrap starting.
I20240502 07:58:22.147398 6648 raft_consensus.cc:374] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:22.147426 6542 raft_consensus.cc:374] T 18a448dbe2af493abefbc728a8f85186 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:22.147431 6648 raft_consensus.cc:729] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5bc7863cd294f77a932c4a40f98b79d, State: Initialized, Role: FOLLOWER
I20240502 07:58:22.147459 6542 raft_consensus.cc:729] T 18a448dbe2af493abefbc728a8f85186 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89bf5c0a6b2a4d949851089446f17b96, State: Initialized, Role: FOLLOWER
I20240502 07:58:22.147532 6648 consensus_queue.cc:260] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } }
I20240502 07:58:22.147568 6542 consensus_queue.cc:260] T 18a448dbe2af493abefbc728a8f85186 P 89bf5c0a6b2a4d949851089446f17b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } }
I20240502 07:58:22.147783 6648 ts_tablet_manager.cc:1398] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:22.147825 6542 ts_tablet_manager.cc:1398] T 18a448dbe2af493abefbc728a8f85186 P 89bf5c0a6b2a4d949851089446f17b96: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:22.147982 6542 tablet_bootstrap.cc:492] T 635b0a1a0fd64eeeac70025014c9ab81 P 89bf5c0a6b2a4d949851089446f17b96: Bootstrap starting.
I20240502 07:58:22.148175 6655 tablet_bootstrap.cc:654] T 5c8c7437c85943ba9113595c17bf6136 P eda55a8566c447cfa749457825b70688: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:22.148746 6542 tablet_bootstrap.cc:654] T 635b0a1a0fd64eeeac70025014c9ab81 P 89bf5c0a6b2a4d949851089446f17b96: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:22.149490 6655 tablet_bootstrap.cc:492] T 5c8c7437c85943ba9113595c17bf6136 P eda55a8566c447cfa749457825b70688: No bootstrap required, opened a new log
I20240502 07:58:22.149567 6655 ts_tablet_manager.cc:1367] T 5c8c7437c85943ba9113595c17bf6136 P eda55a8566c447cfa749457825b70688: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:22.149948 6655 raft_consensus.cc:348] T 5c8c7437c85943ba9113595c17bf6136 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } }
I20240502 07:58:22.150043 6655 raft_consensus.cc:374] T 5c8c7437c85943ba9113595c17bf6136 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:22.150072 6655 raft_consensus.cc:729] T 5c8c7437c85943ba9113595c17bf6136 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eda55a8566c447cfa749457825b70688, State: Initialized, Role: FOLLOWER
I20240502 07:58:22.150169 6655 consensus_queue.cc:260] T 5c8c7437c85943ba9113595c17bf6136 P eda55a8566c447cfa749457825b70688 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } }
I20240502 07:58:22.150231 6529 tablet_bootstrap.cc:492] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841: No bootstrap required, opened a new log
I20240502 07:58:22.150300 6529 ts_tablet_manager.cc:1367] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20240502 07:58:22.150413 6655 ts_tablet_manager.cc:1398] T 5c8c7437c85943ba9113595c17bf6136 P eda55a8566c447cfa749457825b70688: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:22.150694 6529 raft_consensus.cc:348] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } }
I20240502 07:58:22.150801 6529 raft_consensus.cc:374] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:22.150832 6529 raft_consensus.cc:729] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91e82003a656442189a329b29846a841, State: Initialized, Role: FOLLOWER
I20240502 07:58:22.150961 6529 consensus_queue.cc:260] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } }
I20240502 07:58:22.151248 6529 ts_tablet_manager.cc:1398] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:22.151384 6542 tablet_bootstrap.cc:492] T 635b0a1a0fd64eeeac70025014c9ab81 P 89bf5c0a6b2a4d949851089446f17b96: No bootstrap required, opened a new log
I20240502 07:58:22.151402 6529 tablet_bootstrap.cc:492] T 635b0a1a0fd64eeeac70025014c9ab81 P 91e82003a656442189a329b29846a841: Bootstrap starting.
I20240502 07:58:22.151458 6542 ts_tablet_manager.cc:1367] T 635b0a1a0fd64eeeac70025014c9ab81 P 89bf5c0a6b2a4d949851089446f17b96: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20240502 07:58:22.151832 6542 raft_consensus.cc:348] T 635b0a1a0fd64eeeac70025014c9ab81 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } }
I20240502 07:58:22.151940 6542 raft_consensus.cc:374] T 635b0a1a0fd64eeeac70025014c9ab81 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:22.151973 6542 raft_consensus.cc:729] T 635b0a1a0fd64eeeac70025014c9ab81 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89bf5c0a6b2a4d949851089446f17b96, State: Initialized, Role: FOLLOWER
I20240502 07:58:22.152068 6542 consensus_queue.cc:260] T 635b0a1a0fd64eeeac70025014c9ab81 P 89bf5c0a6b2a4d949851089446f17b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } }
I20240502 07:58:22.152312 6542 ts_tablet_manager.cc:1398] T 635b0a1a0fd64eeeac70025014c9ab81 P 89bf5c0a6b2a4d949851089446f17b96: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:22.152468 6529 tablet_bootstrap.cc:654] T 635b0a1a0fd64eeeac70025014c9ab81 P 91e82003a656442189a329b29846a841: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:22.152468 6542 tablet_bootstrap.cc:492] T 5c8c7437c85943ba9113595c17bf6136 P 89bf5c0a6b2a4d949851089446f17b96: Bootstrap starting.
I20240502 07:58:22.152611 6651 tablet_bootstrap.cc:492] T 18a448dbe2af493abefbc728a8f85186 P e020e7ffe8ce423a8bdefec667161217: No bootstrap required, opened a new log
I20240502 07:58:22.152686 6651 ts_tablet_manager.cc:1367] T 18a448dbe2af493abefbc728a8f85186 P e020e7ffe8ce423a8bdefec667161217: Time spent bootstrapping tablet: real 0.022s user 0.001s sys 0.000s
I20240502 07:58:22.153095 6651 raft_consensus.cc:348] T 18a448dbe2af493abefbc728a8f85186 P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } }
I20240502 07:58:22.153199 6651 raft_consensus.cc:374] T 18a448dbe2af493abefbc728a8f85186 P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:22.153210 6542 tablet_bootstrap.cc:654] T 5c8c7437c85943ba9113595c17bf6136 P 89bf5c0a6b2a4d949851089446f17b96: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:22.153228 6651 raft_consensus.cc:729] T 18a448dbe2af493abefbc728a8f85186 P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e020e7ffe8ce423a8bdefec667161217, State: Initialized, Role: FOLLOWER
I20240502 07:58:22.153332 6651 consensus_queue.cc:260] T 18a448dbe2af493abefbc728a8f85186 P e020e7ffe8ce423a8bdefec667161217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } }
I20240502 07:58:22.153589 6651 ts_tablet_manager.cc:1398] T 18a448dbe2af493abefbc728a8f85186 P e020e7ffe8ce423a8bdefec667161217: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:22.153743 6651 tablet_bootstrap.cc:492] T 635b0a1a0fd64eeeac70025014c9ab81 P e020e7ffe8ce423a8bdefec667161217: Bootstrap starting.
I20240502 07:58:22.154484 6651 tablet_bootstrap.cc:654] T 635b0a1a0fd64eeeac70025014c9ab81 P e020e7ffe8ce423a8bdefec667161217: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:22.155357 6651 tablet_bootstrap.cc:492] T 635b0a1a0fd64eeeac70025014c9ab81 P e020e7ffe8ce423a8bdefec667161217: No bootstrap required, opened a new log
I20240502 07:58:22.155448 6651 ts_tablet_manager.cc:1367] T 635b0a1a0fd64eeeac70025014c9ab81 P e020e7ffe8ce423a8bdefec667161217: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:22.156184 6651 raft_consensus.cc:348] T 635b0a1a0fd64eeeac70025014c9ab81 P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } }
I20240502 07:58:22.156283 6651 raft_consensus.cc:374] T 635b0a1a0fd64eeeac70025014c9ab81 P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:22.156312 6651 raft_consensus.cc:729] T 635b0a1a0fd64eeeac70025014c9ab81 P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e020e7ffe8ce423a8bdefec667161217, State: Initialized, Role: FOLLOWER
I20240502 07:58:22.156419 6651 consensus_queue.cc:260] T 635b0a1a0fd64eeeac70025014c9ab81 P e020e7ffe8ce423a8bdefec667161217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } }
I20240502 07:58:22.156677 6651 ts_tablet_manager.cc:1398] T 635b0a1a0fd64eeeac70025014c9ab81 P e020e7ffe8ce423a8bdefec667161217: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:22.157347 6529 tablet_bootstrap.cc:492] T 635b0a1a0fd64eeeac70025014c9ab81 P 91e82003a656442189a329b29846a841: No bootstrap required, opened a new log
I20240502 07:58:22.157423 6529 ts_tablet_manager.cc:1367] T 635b0a1a0fd64eeeac70025014c9ab81 P 91e82003a656442189a329b29846a841: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 07:58:22.157598 6542 tablet_bootstrap.cc:492] T 5c8c7437c85943ba9113595c17bf6136 P 89bf5c0a6b2a4d949851089446f17b96: No bootstrap required, opened a new log
I20240502 07:58:22.157683 6542 ts_tablet_manager.cc:1367] T 5c8c7437c85943ba9113595c17bf6136 P 89bf5c0a6b2a4d949851089446f17b96: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20240502 07:58:22.157807 6529 raft_consensus.cc:348] T 635b0a1a0fd64eeeac70025014c9ab81 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } }
I20240502 07:58:22.157900 6529 raft_consensus.cc:374] T 635b0a1a0fd64eeeac70025014c9ab81 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:22.157930 6529 raft_consensus.cc:729] T 635b0a1a0fd64eeeac70025014c9ab81 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91e82003a656442189a329b29846a841, State: Initialized, Role: FOLLOWER
I20240502 07:58:22.158041 6542 raft_consensus.cc:348] T 5c8c7437c85943ba9113595c17bf6136 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } }
I20240502 07:58:22.158037 6529 consensus_queue.cc:260] T 635b0a1a0fd64eeeac70025014c9ab81 P 91e82003a656442189a329b29846a841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } }
I20240502 07:58:22.158118 6542 raft_consensus.cc:374] T 5c8c7437c85943ba9113595c17bf6136 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:22.158144 6542 raft_consensus.cc:729] T 5c8c7437c85943ba9113595c17bf6136 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89bf5c0a6b2a4d949851089446f17b96, State: Initialized, Role: FOLLOWER
I20240502 07:58:22.158286 6529 ts_tablet_manager.cc:1398] T 635b0a1a0fd64eeeac70025014c9ab81 P 91e82003a656442189a329b29846a841: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:22.158268 6542 consensus_queue.cc:260] T 5c8c7437c85943ba9113595c17bf6136 P 89bf5c0a6b2a4d949851089446f17b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } }
I20240502 07:58:22.158429 6529 tablet_bootstrap.cc:492] T 5c8c7437c85943ba9113595c17bf6136 P 91e82003a656442189a329b29846a841: Bootstrap starting.
I20240502 07:58:22.158510 6542 ts_tablet_manager.cc:1398] T 5c8c7437c85943ba9113595c17bf6136 P 89bf5c0a6b2a4d949851089446f17b96: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:22.159240 6529 tablet_bootstrap.cc:654] T 5c8c7437c85943ba9113595c17bf6136 P 91e82003a656442189a329b29846a841: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:22.160571 6529 tablet_bootstrap.cc:492] T 5c8c7437c85943ba9113595c17bf6136 P 91e82003a656442189a329b29846a841: No bootstrap required, opened a new log
I20240502 07:58:22.160653 6529 ts_tablet_manager.cc:1367] T 5c8c7437c85943ba9113595c17bf6136 P 91e82003a656442189a329b29846a841: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:22.161087 6529 raft_consensus.cc:348] T 5c8c7437c85943ba9113595c17bf6136 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } }
I20240502 07:58:22.161187 6529 raft_consensus.cc:374] T 5c8c7437c85943ba9113595c17bf6136 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:22.161219 6529 raft_consensus.cc:729] T 5c8c7437c85943ba9113595c17bf6136 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91e82003a656442189a329b29846a841, State: Initialized, Role: FOLLOWER
I20240502 07:58:22.161340 6529 consensus_queue.cc:260] T 5c8c7437c85943ba9113595c17bf6136 P 91e82003a656442189a329b29846a841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } }
I20240502 07:58:22.161638 6529 ts_tablet_manager.cc:1398] T 5c8c7437c85943ba9113595c17bf6136 P 91e82003a656442189a329b29846a841: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:22.190428 6545 raft_consensus.cc:482] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:22.190523 6545 raft_consensus.cc:504] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } }
I20240502 07:58:22.190959 6545 leader_election.cc:288] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91e82003a656442189a329b29846a841 (127.3.139.65:43429), e020e7ffe8ce423a8bdefec667161217 (127.3.139.69:39399), 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67:37141), d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70:41047)
I20240502 07:58:22.191184 5628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "635b0a1a0fd64eeeac70025014c9ab81" candidate_uuid: "eda55a8566c447cfa749457825b70688" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e82003a656442189a329b29846a841" is_pre_election: true
I20240502 07:58:22.191296 5628 raft_consensus.cc:2454] T 635b0a1a0fd64eeeac70025014c9ab81 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eda55a8566c447cfa749457825b70688 in term 0.
I20240502 07:58:22.191267 5890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "635b0a1a0fd64eeeac70025014c9ab81" candidate_uuid: "eda55a8566c447cfa749457825b70688" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89bf5c0a6b2a4d949851089446f17b96" is_pre_election: true
I20240502 07:58:22.191388 5890 raft_consensus.cc:2454] T 635b0a1a0fd64eeeac70025014c9ab81 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eda55a8566c447cfa749457825b70688 in term 0.
I20240502 07:58:22.191411 6285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "635b0a1a0fd64eeeac70025014c9ab81" candidate_uuid: "eda55a8566c447cfa749457825b70688" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" is_pre_election: true
I20240502 07:58:22.191516 6285 raft_consensus.cc:2454] T 635b0a1a0fd64eeeac70025014c9ab81 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eda55a8566c447cfa749457825b70688 in term 0.
I20240502 07:58:22.191591 6154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "635b0a1a0fd64eeeac70025014c9ab81" candidate_uuid: "eda55a8566c447cfa749457825b70688" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e020e7ffe8ce423a8bdefec667161217" is_pre_election: true
I20240502 07:58:22.191713 6154 raft_consensus.cc:2454] T 635b0a1a0fd64eeeac70025014c9ab81 P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eda55a8566c447cfa749457825b70688 in term 0.
I20240502 07:58:22.191820 5695 leader_election.cc:302] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 91e82003a656442189a329b29846a841, d58f6d1ff8cf472f8a9edf3700e64bfb, eda55a8566c447cfa749457825b70688; no voters:
I20240502 07:58:22.191983 6545 raft_consensus.cc:2781] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:22.192039 6545 raft_consensus.cc:482] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:22.192075 6545 raft_consensus.cc:3037] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:22.192802 6545 raft_consensus.cc:504] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } }
I20240502 07:58:22.193151 6545 leader_election.cc:288] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688 [CANDIDATE]: Term 1 election: Requested vote from peers 91e82003a656442189a329b29846a841 (127.3.139.65:43429), e020e7ffe8ce423a8bdefec667161217 (127.3.139.69:39399), 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67:37141), d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70:41047)
I20240502 07:58:22.193367 6154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "635b0a1a0fd64eeeac70025014c9ab81" candidate_uuid: "eda55a8566c447cfa749457825b70688" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e020e7ffe8ce423a8bdefec667161217"
I20240502 07:58:22.193435 6285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "635b0a1a0fd64eeeac70025014c9ab81" candidate_uuid: "eda55a8566c447cfa749457825b70688" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb"
I20240502 07:58:22.193434 5890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "635b0a1a0fd64eeeac70025014c9ab81" candidate_uuid: "eda55a8566c447cfa749457825b70688" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89bf5c0a6b2a4d949851089446f17b96"
I20240502 07:58:22.193480 6154 raft_consensus.cc:3037] T 635b0a1a0fd64eeeac70025014c9ab81 P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:22.193511 6285 raft_consensus.cc:3037] T 635b0a1a0fd64eeeac70025014c9ab81 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:22.193522 5890 raft_consensus.cc:3037] T 635b0a1a0fd64eeeac70025014c9ab81 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:22.194119 6285 raft_consensus.cc:2454] T 635b0a1a0fd64eeeac70025014c9ab81 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eda55a8566c447cfa749457825b70688 in term 1.
I20240502 07:58:22.194315 6154 raft_consensus.cc:2454] T 635b0a1a0fd64eeeac70025014c9ab81 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eda55a8566c447cfa749457825b70688 in term 1.
I20240502 07:58:22.194316 5890 raft_consensus.cc:2454] T 635b0a1a0fd64eeeac70025014c9ab81 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eda55a8566c447cfa749457825b70688 in term 1.
I20240502 07:58:22.194360 5628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "635b0a1a0fd64eeeac70025014c9ab81" candidate_uuid: "eda55a8566c447cfa749457825b70688" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e82003a656442189a329b29846a841"
I20240502 07:58:22.194459 5628 raft_consensus.cc:3037] T 635b0a1a0fd64eeeac70025014c9ab81 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:22.194605 5695 leader_election.cc:302] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 89bf5c0a6b2a4d949851089446f17b96, d58f6d1ff8cf472f8a9edf3700e64bfb, eda55a8566c447cfa749457825b70688; no voters:
I20240502 07:58:22.194759 6545 raft_consensus.cc:2781] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:22.194834 6545 raft_consensus.cc:686] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688 [term 1 LEADER]: Becoming Leader. State: Replica: eda55a8566c447cfa749457825b70688, State: Running, Role: LEADER
I20240502 07:58:22.194972 6545 consensus_queue.cc:237] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } }
I20240502 07:58:22.195353 5628 raft_consensus.cc:2454] T 635b0a1a0fd64eeeac70025014c9ab81 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eda55a8566c447cfa749457825b70688 in term 1.
I20240502 07:58:22.195395 6551 raft_consensus.cc:482] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:22.195456 6551 raft_consensus.cc:504] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } }
I20240502 07:58:22.196745 6551 leader_election.cc:288] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eda55a8566c447cfa749457825b70688 (127.3.139.66:39925), 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67:37141), d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68:44287), e020e7ffe8ce423a8bdefec667161217 (127.3.139.69:39399)
I20240502 07:58:22.197081 5488 catalog_manager.cc:5484] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688 reported cstate change: term changed from 0 to 1, leader changed from <none> to eda55a8566c447cfa749457825b70688 (127.3.139.66). New cstate: current_term: 1 leader_uuid: "eda55a8566c447cfa749457825b70688" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:22.201823 5759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18a448dbe2af493abefbc728a8f85186" candidate_uuid: "91e82003a656442189a329b29846a841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eda55a8566c447cfa749457825b70688" is_pre_election: true
I20240502 07:58:22.201934 5759 raft_consensus.cc:2454] T 18a448dbe2af493abefbc728a8f85186 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91e82003a656442189a329b29846a841 in term 0.
I20240502 07:58:22.202369 6023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18a448dbe2af493abefbc728a8f85186" candidate_uuid: "91e82003a656442189a329b29846a841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5bc7863cd294f77a932c4a40f98b79d" is_pre_election: true
I20240502 07:58:22.202473 6023 raft_consensus.cc:2454] T 18a448dbe2af493abefbc728a8f85186 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91e82003a656442189a329b29846a841 in term 0.
I20240502 07:58:22.202808 6154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18a448dbe2af493abefbc728a8f85186" candidate_uuid: "91e82003a656442189a329b29846a841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e020e7ffe8ce423a8bdefec667161217" is_pre_election: true
I20240502 07:58:22.202915 6154 raft_consensus.cc:2454] T 18a448dbe2af493abefbc728a8f85186 P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91e82003a656442189a329b29846a841 in term 0.
I20240502 07:58:22.203200 5565 leader_election.cc:302] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 91e82003a656442189a329b29846a841, d5bc7863cd294f77a932c4a40f98b79d, e020e7ffe8ce423a8bdefec667161217; no voters:
I20240502 07:58:22.203437 6551 raft_consensus.cc:2781] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:22.203508 6551 raft_consensus.cc:482] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:22.203552 6551 raft_consensus.cc:3037] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:22.203896 5890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18a448dbe2af493abefbc728a8f85186" candidate_uuid: "91e82003a656442189a329b29846a841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89bf5c0a6b2a4d949851089446f17b96" is_pre_election: true
I20240502 07:58:22.204007 5890 raft_consensus.cc:2454] T 18a448dbe2af493abefbc728a8f85186 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91e82003a656442189a329b29846a841 in term 0.
I20240502 07:58:22.204258 6551 raft_consensus.cc:504] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } }
I20240502 07:58:22.204613 6551 leader_election.cc:288] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [CANDIDATE]: Term 1 election: Requested vote from peers eda55a8566c447cfa749457825b70688 (127.3.139.66:39925), 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67:37141), d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68:44287), e020e7ffe8ce423a8bdefec667161217 (127.3.139.69:39399)
I20240502 07:58:22.204921 6023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18a448dbe2af493abefbc728a8f85186" candidate_uuid: "91e82003a656442189a329b29846a841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5bc7863cd294f77a932c4a40f98b79d"
I20240502 07:58:22.204929 5759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18a448dbe2af493abefbc728a8f85186" candidate_uuid: "91e82003a656442189a329b29846a841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eda55a8566c447cfa749457825b70688"
I20240502 07:58:22.205015 5759 raft_consensus.cc:3037] T 18a448dbe2af493abefbc728a8f85186 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:22.205015 6023 raft_consensus.cc:3037] T 18a448dbe2af493abefbc728a8f85186 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:22.205839 6023 raft_consensus.cc:2454] T 18a448dbe2af493abefbc728a8f85186 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91e82003a656442189a329b29846a841 in term 1.
I20240502 07:58:22.205839 5759 raft_consensus.cc:2454] T 18a448dbe2af493abefbc728a8f85186 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91e82003a656442189a329b29846a841 in term 1.
I20240502 07:58:22.206056 6154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18a448dbe2af493abefbc728a8f85186" candidate_uuid: "91e82003a656442189a329b29846a841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e020e7ffe8ce423a8bdefec667161217"
I20240502 07:58:22.206146 6154 raft_consensus.cc:3037] T 18a448dbe2af493abefbc728a8f85186 P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:22.206122 5564 leader_election.cc:302] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 91e82003a656442189a329b29846a841, d5bc7863cd294f77a932c4a40f98b79d, eda55a8566c447cfa749457825b70688; no voters:
I20240502 07:58:22.206282 6551 raft_consensus.cc:2781] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:22.206444 6551 raft_consensus.cc:686] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [term 1 LEADER]: Becoming Leader. State: Replica: 91e82003a656442189a329b29846a841, State: Running, Role: LEADER
I20240502 07:58:22.206691 6551 consensus_queue.cc:237] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } }
I20240502 07:58:22.206846 6154 raft_consensus.cc:2454] T 18a448dbe2af493abefbc728a8f85186 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91e82003a656442189a329b29846a841 in term 1.
I20240502 07:58:22.207075 5890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18a448dbe2af493abefbc728a8f85186" candidate_uuid: "91e82003a656442189a329b29846a841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89bf5c0a6b2a4d949851089446f17b96"
I20240502 07:58:22.207154 5890 raft_consensus.cc:3037] T 18a448dbe2af493abefbc728a8f85186 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:22.207769 5890 raft_consensus.cc:2454] T 18a448dbe2af493abefbc728a8f85186 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91e82003a656442189a329b29846a841 in term 1.
I20240502 07:58:22.208900 5488 catalog_manager.cc:5484] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91e82003a656442189a329b29846a841 (127.3.139.65). New cstate: current_term: 1 leader_uuid: "91e82003a656442189a329b29846a841" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:22.211671 6560 raft_consensus.cc:482] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:22.211747 6560 raft_consensus.cc:504] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } }
I20240502 07:58:22.212095 6560 leader_election.cc:288] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eda55a8566c447cfa749457825b70688 (127.3.139.66:39925), 91e82003a656442189a329b29846a841 (127.3.139.65:43429), d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70:41047), 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67:37141)
I20240502 07:58:22.212324 5759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c8c7437c85943ba9113595c17bf6136" candidate_uuid: "d5bc7863cd294f77a932c4a40f98b79d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eda55a8566c447cfa749457825b70688" is_pre_election: true
I20240502 07:58:22.212360 6285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c8c7437c85943ba9113595c17bf6136" candidate_uuid: "d5bc7863cd294f77a932c4a40f98b79d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" is_pre_election: true
I20240502 07:58:22.212400 5890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c8c7437c85943ba9113595c17bf6136" candidate_uuid: "d5bc7863cd294f77a932c4a40f98b79d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89bf5c0a6b2a4d949851089446f17b96" is_pre_election: true
I20240502 07:58:22.212436 5759 raft_consensus.cc:2454] T 5c8c7437c85943ba9113595c17bf6136 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5bc7863cd294f77a932c4a40f98b79d in term 0.
I20240502 07:58:22.212493 5890 raft_consensus.cc:2454] T 5c8c7437c85943ba9113595c17bf6136 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5bc7863cd294f77a932c4a40f98b79d in term 0.
I20240502 07:58:22.212491 6285 raft_consensus.cc:2454] T 5c8c7437c85943ba9113595c17bf6136 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5bc7863cd294f77a932c4a40f98b79d in term 0.
I20240502 07:58:22.212635 5628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c8c7437c85943ba9113595c17bf6136" candidate_uuid: "d5bc7863cd294f77a932c4a40f98b79d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e82003a656442189a329b29846a841" is_pre_election: true
I20240502 07:58:22.212728 5628 raft_consensus.cc:2454] T 5c8c7437c85943ba9113595c17bf6136 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5bc7863cd294f77a932c4a40f98b79d in term 0.
I20240502 07:58:22.212723 5958 leader_election.cc:302] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: d58f6d1ff8cf472f8a9edf3700e64bfb, d5bc7863cd294f77a932c4a40f98b79d, eda55a8566c447cfa749457825b70688; no voters:
I20240502 07:58:22.212874 6560 raft_consensus.cc:2781] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:22.212946 6560 raft_consensus.cc:482] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:22.212983 6560 raft_consensus.cc:3037] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:22.213586 6560 raft_consensus.cc:504] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } }
I20240502 07:58:22.213981 6560 leader_election.cc:288] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d [CANDIDATE]: Term 1 election: Requested vote from peers eda55a8566c447cfa749457825b70688 (127.3.139.66:39925), 91e82003a656442189a329b29846a841 (127.3.139.65:43429), d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70:41047), 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67:37141)
I20240502 07:58:22.214179 5759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c8c7437c85943ba9113595c17bf6136" candidate_uuid: "d5bc7863cd294f77a932c4a40f98b79d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eda55a8566c447cfa749457825b70688"
I20240502 07:58:22.214241 5890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c8c7437c85943ba9113595c17bf6136" candidate_uuid: "d5bc7863cd294f77a932c4a40f98b79d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89bf5c0a6b2a4d949851089446f17b96"
I20240502 07:58:22.214267 5759 raft_consensus.cc:3037] T 5c8c7437c85943ba9113595c17bf6136 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:22.214324 5890 raft_consensus.cc:3037] T 5c8c7437c85943ba9113595c17bf6136 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:22.214319 6285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c8c7437c85943ba9113595c17bf6136" candidate_uuid: "d5bc7863cd294f77a932c4a40f98b79d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb"
I20240502 07:58:22.214404 6285 raft_consensus.cc:3037] T 5c8c7437c85943ba9113595c17bf6136 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:22.215026 6285 raft_consensus.cc:2454] T 5c8c7437c85943ba9113595c17bf6136 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5bc7863cd294f77a932c4a40f98b79d in term 1.
I20240502 07:58:22.215025 5759 raft_consensus.cc:2454] T 5c8c7437c85943ba9113595c17bf6136 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5bc7863cd294f77a932c4a40f98b79d in term 1.
I20240502 07:58:22.215025 5890 raft_consensus.cc:2454] T 5c8c7437c85943ba9113595c17bf6136 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5bc7863cd294f77a932c4a40f98b79d in term 1.
I20240502 07:58:22.215248 5628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c8c7437c85943ba9113595c17bf6136" candidate_uuid: "d5bc7863cd294f77a932c4a40f98b79d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e82003a656442189a329b29846a841"
I20240502 07:58:22.215349 5628 raft_consensus.cc:3037] T 5c8c7437c85943ba9113595c17bf6136 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:22.215346 5957 leader_election.cc:302] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 89bf5c0a6b2a4d949851089446f17b96, d5bc7863cd294f77a932c4a40f98b79d, eda55a8566c447cfa749457825b70688; no voters:
I20240502 07:58:22.215572 6560 raft_consensus.cc:2781] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:22.215654 6560 raft_consensus.cc:686] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d [term 1 LEADER]: Becoming Leader. State: Replica: d5bc7863cd294f77a932c4a40f98b79d, State: Running, Role: LEADER
I20240502 07:58:22.215777 6560 consensus_queue.cc:237] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } }
I20240502 07:58:22.216027 5628 raft_consensus.cc:2454] T 5c8c7437c85943ba9113595c17bf6136 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5bc7863cd294f77a932c4a40f98b79d in term 1.
I20240502 07:58:22.217337 5488 catalog_manager.cc:5484] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d reported cstate change: term changed from 0 to 1, leader changed from <none> to d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68). New cstate: current_term: 1 leader_uuid: "d5bc7863cd294f77a932c4a40f98b79d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } health_report { overall_health: HEALTHY } } }
W20240502 07:58:22.337824 6743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:22.338027 6743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:22.340973 6743 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_2'
I20240502 07:58:22.417045 5759 raft_consensus.cc:1261] T 18a448dbe2af493abefbc728a8f85186 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:22.417215 6023 raft_consensus.cc:1261] T 18a448dbe2af493abefbc728a8f85186 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:22.417636 6770 consensus_queue.cc:1035] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:22.418004 5890 raft_consensus.cc:1261] T 18a448dbe2af493abefbc728a8f85186 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:22.418049 6770 consensus_queue.cc:1035] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:22.418805 6770 consensus_queue.cc:1035] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:22.420358 6154 raft_consensus.cc:1261] T 18a448dbe2af493abefbc728a8f85186 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:22.421073 5628 raft_consensus.cc:1261] T 635b0a1a0fd64eeeac70025014c9ab81 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer eda55a8566c447cfa749457825b70688: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:22.422233 6285 raft_consensus.cc:1261] T 635b0a1a0fd64eeeac70025014c9ab81 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Refusing update from remote peer eda55a8566c447cfa749457825b70688: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:22.422400 6154 raft_consensus.cc:1261] T 635b0a1a0fd64eeeac70025014c9ab81 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer eda55a8566c447cfa749457825b70688: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:22.422570 6540 consensus_queue.cc:1035] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:22.422753 6545 consensus_queue.cc:1035] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:22.425848 6551 consensus_queue.cc:1035] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:22.427211 6285 raft_consensus.cc:1261] T 5c8c7437c85943ba9113595c17bf6136 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:22.429795 5892 raft_consensus.cc:1261] T 5c8c7437c85943ba9113595c17bf6136 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:22.429955 5890 raft_consensus.cc:1261] T 635b0a1a0fd64eeeac70025014c9ab81 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer eda55a8566c447cfa749457825b70688: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:22.430220 6545 consensus_queue.cc:1035] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:22.430423 5759 raft_consensus.cc:1261] T 5c8c7437c85943ba9113595c17bf6136 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:22.430974 5628 raft_consensus.cc:1261] T 5c8c7437c85943ba9113595c17bf6136 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:22.430938 6774 consensus_queue.cc:1035] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:22.431385 6774 consensus_queue.cc:1035] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:22.431628 6774 consensus_queue.cc:1035] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:22.433848 6545 consensus_queue.cc:1035] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:22.435714 6515 consensus_queue.cc:1035] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 138.899 ms
time per row: 0.0694493 ms
I20240502 07:58:22.523144 5481 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40652:
name: "rebalance_test_table_3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 5
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:58:22.528601 5740 tablet_service.cc:1467] Processing CreateTablet for tablet 96dcb06388374514951f24393f288eca (DEFAULT_TABLE table=rebalance_test_table_3 [id=d0a0bf64867c4915ad797ab2bcd7101f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:22.528899 5740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96dcb06388374514951f24393f288eca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:22.529281 5739 tablet_service.cc:1467] Processing CreateTablet for tablet bc41794fb8c545149694aa84dde1cdb0 (DEFAULT_TABLE table=rebalance_test_table_3 [id=d0a0bf64867c4915ad797ab2bcd7101f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:22.529539 5739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc41794fb8c545149694aa84dde1cdb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:22.530876 6655 tablet_bootstrap.cc:492] T 96dcb06388374514951f24393f288eca P eda55a8566c447cfa749457825b70688: Bootstrap starting.
I20240502 07:58:22.531116 5610 tablet_service.cc:1467] Processing CreateTablet for tablet 96dcb06388374514951f24393f288eca (DEFAULT_TABLE table=rebalance_test_table_3 [id=d0a0bf64867c4915ad797ab2bcd7101f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:22.531356 5610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96dcb06388374514951f24393f288eca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:22.531666 6655 tablet_bootstrap.cc:654] T 96dcb06388374514951f24393f288eca P eda55a8566c447cfa749457825b70688: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:22.532084 5872 tablet_service.cc:1467] Processing CreateTablet for tablet bc41794fb8c545149694aa84dde1cdb0 (DEFAULT_TABLE table=rebalance_test_table_3 [id=d0a0bf64867c4915ad797ab2bcd7101f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:22.532326 5872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc41794fb8c545149694aa84dde1cdb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:22.533330 6529 tablet_bootstrap.cc:492] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841: Bootstrap starting.
I20240502 07:58:22.533727 6003 tablet_service.cc:1467] Processing CreateTablet for tablet 96dcb06388374514951f24393f288eca (DEFAULT_TABLE table=rebalance_test_table_3 [id=d0a0bf64867c4915ad797ab2bcd7101f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:22.533999 6003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96dcb06388374514951f24393f288eca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:22.534154 6529 tablet_bootstrap.cc:654] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:22.534260 6001 tablet_service.cc:1467] Processing CreateTablet for tablet 2a3919f144ed4a27b03e7c4a251499d4 (DEFAULT_TABLE table=rebalance_test_table_3 [id=d0a0bf64867c4915ad797ab2bcd7101f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:22.534489 6001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a3919f144ed4a27b03e7c4a251499d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:22.534546 6264 tablet_service.cc:1467] Processing CreateTablet for tablet 2a3919f144ed4a27b03e7c4a251499d4 (DEFAULT_TABLE table=rebalance_test_table_3 [id=d0a0bf64867c4915ad797ab2bcd7101f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:22.534785 6264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a3919f144ed4a27b03e7c4a251499d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:22.534842 6002 tablet_service.cc:1467] Processing CreateTablet for tablet bc41794fb8c545149694aa84dde1cdb0 (DEFAULT_TABLE table=rebalance_test_table_3 [id=d0a0bf64867c4915ad797ab2bcd7101f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:22.535045 6002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc41794fb8c545149694aa84dde1cdb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:22.535176 6542 tablet_bootstrap.cc:492] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96: Bootstrap starting.
I20240502 07:58:22.535990 6542 tablet_bootstrap.cc:654] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:22.536170 6265 tablet_service.cc:1467] Processing CreateTablet for tablet 96dcb06388374514951f24393f288eca (DEFAULT_TABLE table=rebalance_test_table_3 [id=d0a0bf64867c4915ad797ab2bcd7101f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:22.536393 6265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96dcb06388374514951f24393f288eca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:22.537276 6648 tablet_bootstrap.cc:492] T 96dcb06388374514951f24393f288eca P d5bc7863cd294f77a932c4a40f98b79d: Bootstrap starting.
I20240502 07:58:22.538091 6648 tablet_bootstrap.cc:654] T 96dcb06388374514951f24393f288eca P d5bc7863cd294f77a932c4a40f98b79d: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:22.538204 6263 tablet_service.cc:1467] Processing CreateTablet for tablet bc41794fb8c545149694aa84dde1cdb0 (DEFAULT_TABLE table=rebalance_test_table_3 [id=d0a0bf64867c4915ad797ab2bcd7101f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:22.538420 6263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc41794fb8c545149694aa84dde1cdb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:22.538902 5739 tablet_service.cc:1467] Processing CreateTablet for tablet 2a3919f144ed4a27b03e7c4a251499d4 (DEFAULT_TABLE table=rebalance_test_table_3 [id=d0a0bf64867c4915ad797ab2bcd7101f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:22.538916 6132 tablet_service.cc:1467] Processing CreateTablet for tablet 96dcb06388374514951f24393f288eca (DEFAULT_TABLE table=rebalance_test_table_3 [id=d0a0bf64867c4915ad797ab2bcd7101f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:22.539134 5739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a3919f144ed4a27b03e7c4a251499d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:22.539285 6133 tablet_service.cc:1467] Processing CreateTablet for tablet bc41794fb8c545149694aa84dde1cdb0 (DEFAULT_TABLE table=rebalance_test_table_3 [id=d0a0bf64867c4915ad797ab2bcd7101f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:22.539497 6133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc41794fb8c545149694aa84dde1cdb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:22.540071 6650 tablet_bootstrap.cc:492] T 2a3919f144ed4a27b03e7c4a251499d4 P d58f6d1ff8cf472f8a9edf3700e64bfb: Bootstrap starting.
I20240502 07:58:22.540143 5610 tablet_service.cc:1467] Processing CreateTablet for tablet 2a3919f144ed4a27b03e7c4a251499d4 (DEFAULT_TABLE table=rebalance_test_table_3 [id=d0a0bf64867c4915ad797ab2bcd7101f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:22.540349 5610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a3919f144ed4a27b03e7c4a251499d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:22.540951 6650 tablet_bootstrap.cc:654] T 2a3919f144ed4a27b03e7c4a251499d4 P d58f6d1ff8cf472f8a9edf3700e64bfb: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:22.541528 6134 tablet_service.cc:1467] Processing CreateTablet for tablet 2a3919f144ed4a27b03e7c4a251499d4 (DEFAULT_TABLE table=rebalance_test_table_3 [id=d0a0bf64867c4915ad797ab2bcd7101f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:22.541721 6134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a3919f144ed4a27b03e7c4a251499d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:22.543457 6651 tablet_bootstrap.cc:492] T bc41794fb8c545149694aa84dde1cdb0 P e020e7ffe8ce423a8bdefec667161217: Bootstrap starting.
I20240502 07:58:22.543730 6542 tablet_bootstrap.cc:492] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96: No bootstrap required, opened a new log
I20240502 07:58:22.543797 6542 ts_tablet_manager.cc:1367] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.001s
I20240502 07:58:22.543884 6648 tablet_bootstrap.cc:492] T 96dcb06388374514951f24393f288eca P d5bc7863cd294f77a932c4a40f98b79d: No bootstrap required, opened a new log
I20240502 07:58:22.543958 6648 ts_tablet_manager.cc:1367] T 96dcb06388374514951f24393f288eca P d5bc7863cd294f77a932c4a40f98b79d: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240502 07:58:22.544203 6542 raft_consensus.cc:348] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } }
I20240502 07:58:22.544282 6651 tablet_bootstrap.cc:654] T bc41794fb8c545149694aa84dde1cdb0 P e020e7ffe8ce423a8bdefec667161217: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:22.544307 6542 raft_consensus.cc:374] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:22.544334 6542 raft_consensus.cc:729] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89bf5c0a6b2a4d949851089446f17b96, State: Initialized, Role: FOLLOWER
I20240502 07:58:22.544370 6648 raft_consensus.cc:348] T 96dcb06388374514951f24393f288eca P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } }
I20240502 07:58:22.544473 6648 raft_consensus.cc:374] T 96dcb06388374514951f24393f288eca P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:22.544500 6648 raft_consensus.cc:729] T 96dcb06388374514951f24393f288eca P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5bc7863cd294f77a932c4a40f98b79d, State: Initialized, Role: FOLLOWER
I20240502 07:58:22.544472 6542 consensus_queue.cc:260] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } }
I20240502 07:58:22.544610 6648 consensus_queue.cc:260] T 96dcb06388374514951f24393f288eca P d5bc7863cd294f77a932c4a40f98b79d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } }
I20240502 07:58:22.544728 6542 ts_tablet_manager.cc:1398] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240502 07:58:22.544876 6648 ts_tablet_manager.cc:1398] T 96dcb06388374514951f24393f288eca P d5bc7863cd294f77a932c4a40f98b79d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:22.545037 6648 tablet_bootstrap.cc:492] T bc41794fb8c545149694aa84dde1cdb0 P d5bc7863cd294f77a932c4a40f98b79d: Bootstrap starting.
I20240502 07:58:22.545497 6132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96dcb06388374514951f24393f288eca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:22.545817 6648 tablet_bootstrap.cc:654] T bc41794fb8c545149694aa84dde1cdb0 P d5bc7863cd294f77a932c4a40f98b79d: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:22.547247 6648 tablet_bootstrap.cc:492] T bc41794fb8c545149694aa84dde1cdb0 P d5bc7863cd294f77a932c4a40f98b79d: No bootstrap required, opened a new log
I20240502 07:58:22.547322 6648 ts_tablet_manager.cc:1367] T bc41794fb8c545149694aa84dde1cdb0 P d5bc7863cd294f77a932c4a40f98b79d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:22.547719 6648 raft_consensus.cc:348] T bc41794fb8c545149694aa84dde1cdb0 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } }
I20240502 07:58:22.547828 6648 raft_consensus.cc:374] T bc41794fb8c545149694aa84dde1cdb0 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:22.547859 6648 raft_consensus.cc:729] T bc41794fb8c545149694aa84dde1cdb0 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5bc7863cd294f77a932c4a40f98b79d, State: Initialized, Role: FOLLOWER
I20240502 07:58:22.547966 6648 consensus_queue.cc:260] T bc41794fb8c545149694aa84dde1cdb0 P d5bc7863cd294f77a932c4a40f98b79d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } }
I20240502 07:58:22.548218 6648 ts_tablet_manager.cc:1398] T bc41794fb8c545149694aa84dde1cdb0 P d5bc7863cd294f77a932c4a40f98b79d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:22.548398 6648 tablet_bootstrap.cc:492] T 2a3919f144ed4a27b03e7c4a251499d4 P d5bc7863cd294f77a932c4a40f98b79d: Bootstrap starting.
I20240502 07:58:22.548626 6651 tablet_bootstrap.cc:492] T bc41794fb8c545149694aa84dde1cdb0 P e020e7ffe8ce423a8bdefec667161217: No bootstrap required, opened a new log
I20240502 07:58:22.548696 6651 ts_tablet_manager.cc:1367] T bc41794fb8c545149694aa84dde1cdb0 P e020e7ffe8ce423a8bdefec667161217: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20240502 07:58:22.549368 6651 raft_consensus.cc:348] T bc41794fb8c545149694aa84dde1cdb0 P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } }
I20240502 07:58:22.549480 6648 tablet_bootstrap.cc:654] T 2a3919f144ed4a27b03e7c4a251499d4 P d5bc7863cd294f77a932c4a40f98b79d: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:22.549479 6651 raft_consensus.cc:374] T bc41794fb8c545149694aa84dde1cdb0 P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:22.549508 6651 raft_consensus.cc:729] T bc41794fb8c545149694aa84dde1cdb0 P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e020e7ffe8ce423a8bdefec667161217, State: Initialized, Role: FOLLOWER
I20240502 07:58:22.549638 6651 consensus_queue.cc:260] T bc41794fb8c545149694aa84dde1cdb0 P e020e7ffe8ce423a8bdefec667161217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } }
I20240502 07:58:22.549896 6651 ts_tablet_manager.cc:1398] T bc41794fb8c545149694aa84dde1cdb0 P e020e7ffe8ce423a8bdefec667161217: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:22.550045 6651 tablet_bootstrap.cc:492] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217: Bootstrap starting.
I20240502 07:58:22.550421 6648 tablet_bootstrap.cc:492] T 2a3919f144ed4a27b03e7c4a251499d4 P d5bc7863cd294f77a932c4a40f98b79d: No bootstrap required, opened a new log
I20240502 07:58:22.550469 6655 tablet_bootstrap.cc:492] T 96dcb06388374514951f24393f288eca P eda55a8566c447cfa749457825b70688: No bootstrap required, opened a new log
I20240502 07:58:22.550482 6648 ts_tablet_manager.cc:1367] T 2a3919f144ed4a27b03e7c4a251499d4 P d5bc7863cd294f77a932c4a40f98b79d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:22.550544 6655 ts_tablet_manager.cc:1367] T 96dcb06388374514951f24393f288eca P eda55a8566c447cfa749457825b70688: Time spent bootstrapping tablet: real 0.020s user 0.001s sys 0.000s
I20240502 07:58:22.550843 6651 tablet_bootstrap.cc:654] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:22.550829 6648 raft_consensus.cc:348] T 2a3919f144ed4a27b03e7c4a251499d4 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:22.550912 6648 raft_consensus.cc:374] T 2a3919f144ed4a27b03e7c4a251499d4 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:22.550937 6648 raft_consensus.cc:729] T 2a3919f144ed4a27b03e7c4a251499d4 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5bc7863cd294f77a932c4a40f98b79d, State: Initialized, Role: FOLLOWER
I20240502 07:58:22.550942 6655 raft_consensus.cc:348] T 96dcb06388374514951f24393f288eca P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } }
I20240502 07:58:22.551043 6655 raft_consensus.cc:374] T 96dcb06388374514951f24393f288eca P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:22.551070 6648 consensus_queue.cc:260] T 2a3919f144ed4a27b03e7c4a251499d4 P d5bc7863cd294f77a932c4a40f98b79d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:22.551131 6655 raft_consensus.cc:729] T 96dcb06388374514951f24393f288eca P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eda55a8566c447cfa749457825b70688, State: Initialized, Role: FOLLOWER
I20240502 07:58:22.551239 6655 consensus_queue.cc:260] T 96dcb06388374514951f24393f288eca P eda55a8566c447cfa749457825b70688 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } }
I20240502 07:58:22.551348 6648 ts_tablet_manager.cc:1398] T 2a3919f144ed4a27b03e7c4a251499d4 P d5bc7863cd294f77a932c4a40f98b79d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:22.551517 6655 ts_tablet_manager.cc:1398] T 96dcb06388374514951f24393f288eca P eda55a8566c447cfa749457825b70688: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:22.551678 6655 tablet_bootstrap.cc:492] T bc41794fb8c545149694aa84dde1cdb0 P eda55a8566c447cfa749457825b70688: Bootstrap starting.
I20240502 07:58:22.552488 6655 tablet_bootstrap.cc:654] T bc41794fb8c545149694aa84dde1cdb0 P eda55a8566c447cfa749457825b70688: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:22.554430 6529 tablet_bootstrap.cc:492] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841: No bootstrap required, opened a new log
I20240502 07:58:22.554517 6529 ts_tablet_manager.cc:1367] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.000s
I20240502 07:58:22.554636 6650 tablet_bootstrap.cc:492] T 2a3919f144ed4a27b03e7c4a251499d4 P d58f6d1ff8cf472f8a9edf3700e64bfb: No bootstrap required, opened a new log
I20240502 07:58:22.554677 6651 tablet_bootstrap.cc:492] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217: No bootstrap required, opened a new log
I20240502 07:58:22.554721 6650 ts_tablet_manager.cc:1367] T 2a3919f144ed4a27b03e7c4a251499d4 P d58f6d1ff8cf472f8a9edf3700e64bfb: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.000s
I20240502 07:58:22.554734 6651 ts_tablet_manager.cc:1367] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 07:58:22.554872 6655 tablet_bootstrap.cc:492] T bc41794fb8c545149694aa84dde1cdb0 P eda55a8566c447cfa749457825b70688: No bootstrap required, opened a new log
I20240502 07:58:22.554939 6655 ts_tablet_manager.cc:1367] T bc41794fb8c545149694aa84dde1cdb0 P eda55a8566c447cfa749457825b70688: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240502 07:58:22.555107 6651 raft_consensus.cc:348] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:22.555107 6650 raft_consensus.cc:348] T 2a3919f144ed4a27b03e7c4a251499d4 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:22.555209 6651 raft_consensus.cc:374] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:22.555210 6650 raft_consensus.cc:374] T 2a3919f144ed4a27b03e7c4a251499d4 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:22.555240 6651 raft_consensus.cc:729] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e020e7ffe8ce423a8bdefec667161217, State: Initialized, Role: FOLLOWER
I20240502 07:58:22.555240 6650 raft_consensus.cc:729] T 2a3919f144ed4a27b03e7c4a251499d4 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d58f6d1ff8cf472f8a9edf3700e64bfb, State: Initialized, Role: FOLLOWER
I20240502 07:58:22.555344 6651 consensus_queue.cc:260] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:22.555346 6650 consensus_queue.cc:260] T 2a3919f144ed4a27b03e7c4a251499d4 P d58f6d1ff8cf472f8a9edf3700e64bfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:22.555321 6655 raft_consensus.cc:348] T bc41794fb8c545149694aa84dde1cdb0 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } }
I20240502 07:58:22.555513 6655 raft_consensus.cc:374] T bc41794fb8c545149694aa84dde1cdb0 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:22.555555 6655 raft_consensus.cc:729] T bc41794fb8c545149694aa84dde1cdb0 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eda55a8566c447cfa749457825b70688, State: Initialized, Role: FOLLOWER
I20240502 07:58:22.555588 6650 ts_tablet_manager.cc:1398] T 2a3919f144ed4a27b03e7c4a251499d4 P d58f6d1ff8cf472f8a9edf3700e64bfb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:22.555588 6651 ts_tablet_manager.cc:1398] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:22.555737 6650 tablet_bootstrap.cc:492] T 96dcb06388374514951f24393f288eca P d58f6d1ff8cf472f8a9edf3700e64bfb: Bootstrap starting.
I20240502 07:58:22.555732 6651 tablet_bootstrap.cc:492] T 96dcb06388374514951f24393f288eca P e020e7ffe8ce423a8bdefec667161217: Bootstrap starting.
I20240502 07:58:22.556522 6650 tablet_bootstrap.cc:654] T 96dcb06388374514951f24393f288eca P d58f6d1ff8cf472f8a9edf3700e64bfb: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:22.556522 6651 tablet_bootstrap.cc:654] T 96dcb06388374514951f24393f288eca P e020e7ffe8ce423a8bdefec667161217: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:22.554967 6529 raft_consensus.cc:348] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } }
I20240502 07:58:22.556588 6529 raft_consensus.cc:374] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:22.556631 6529 raft_consensus.cc:729] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91e82003a656442189a329b29846a841, State: Initialized, Role: FOLLOWER
I20240502 07:58:22.556746 6529 consensus_queue.cc:260] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } }
I20240502 07:58:22.557047 6529 ts_tablet_manager.cc:1398] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:22.557202 6529 tablet_bootstrap.cc:492] T 2a3919f144ed4a27b03e7c4a251499d4 P 91e82003a656442189a329b29846a841: Bootstrap starting.
I20240502 07:58:22.557700 6655 consensus_queue.cc:260] T bc41794fb8c545149694aa84dde1cdb0 P eda55a8566c447cfa749457825b70688 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } }
I20240502 07:58:22.558059 6529 tablet_bootstrap.cc:654] T 2a3919f144ed4a27b03e7c4a251499d4 P 91e82003a656442189a329b29846a841: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:22.558787 6655 ts_tablet_manager.cc:1398] T bc41794fb8c545149694aa84dde1cdb0 P eda55a8566c447cfa749457825b70688: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20240502 07:58:22.559373 6529 tablet_bootstrap.cc:492] T 2a3919f144ed4a27b03e7c4a251499d4 P 91e82003a656442189a329b29846a841: No bootstrap required, opened a new log
I20240502 07:58:22.559513 6650 tablet_bootstrap.cc:492] T 96dcb06388374514951f24393f288eca P d58f6d1ff8cf472f8a9edf3700e64bfb: No bootstrap required, opened a new log
I20240502 07:58:22.559593 6650 ts_tablet_manager.cc:1367] T 96dcb06388374514951f24393f288eca P d58f6d1ff8cf472f8a9edf3700e64bfb: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:58:22.559921 6529 ts_tablet_manager.cc:1367] T 2a3919f144ed4a27b03e7c4a251499d4 P 91e82003a656442189a329b29846a841: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:22.560180 6650 raft_consensus.cc:348] T 96dcb06388374514951f24393f288eca P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } }
I20240502 07:58:22.560319 6650 raft_consensus.cc:374] T 96dcb06388374514951f24393f288eca P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:22.560367 6650 raft_consensus.cc:729] T 96dcb06388374514951f24393f288eca P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d58f6d1ff8cf472f8a9edf3700e64bfb, State: Initialized, Role: FOLLOWER
I20240502 07:58:22.560503 6650 consensus_queue.cc:260] T 96dcb06388374514951f24393f288eca P d58f6d1ff8cf472f8a9edf3700e64bfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } }
I20240502 07:58:22.560586 6655 tablet_bootstrap.cc:492] T 2a3919f144ed4a27b03e7c4a251499d4 P eda55a8566c447cfa749457825b70688: Bootstrap starting.
I20240502 07:58:22.560602 6529 raft_consensus.cc:348] T 2a3919f144ed4a27b03e7c4a251499d4 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:22.560755 6529 raft_consensus.cc:374] T 2a3919f144ed4a27b03e7c4a251499d4 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:22.560770 6650 ts_tablet_manager.cc:1398] T 96dcb06388374514951f24393f288eca P d58f6d1ff8cf472f8a9edf3700e64bfb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:22.560855 6529 raft_consensus.cc:729] T 2a3919f144ed4a27b03e7c4a251499d4 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91e82003a656442189a329b29846a841, State: Initialized, Role: FOLLOWER
I20240502 07:58:22.561030 6529 consensus_queue.cc:260] T 2a3919f144ed4a27b03e7c4a251499d4 P 91e82003a656442189a329b29846a841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:22.561395 6651 tablet_bootstrap.cc:492] T 96dcb06388374514951f24393f288eca P e020e7ffe8ce423a8bdefec667161217: No bootstrap required, opened a new log
I20240502 07:58:22.561400 6529 ts_tablet_manager.cc:1398] T 2a3919f144ed4a27b03e7c4a251499d4 P 91e82003a656442189a329b29846a841: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:22.561481 6651 ts_tablet_manager.cc:1367] T 96dcb06388374514951f24393f288eca P e020e7ffe8ce423a8bdefec667161217: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240502 07:58:22.561883 6651 raft_consensus.cc:348] T 96dcb06388374514951f24393f288eca P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } }
I20240502 07:58:22.561990 6651 raft_consensus.cc:374] T 96dcb06388374514951f24393f288eca P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:22.562028 6651 raft_consensus.cc:729] T 96dcb06388374514951f24393f288eca P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e020e7ffe8ce423a8bdefec667161217, State: Initialized, Role: FOLLOWER
I20240502 07:58:22.562057 6650 tablet_bootstrap.cc:492] T bc41794fb8c545149694aa84dde1cdb0 P d58f6d1ff8cf472f8a9edf3700e64bfb: Bootstrap starting.
I20240502 07:58:22.562144 6651 consensus_queue.cc:260] T 96dcb06388374514951f24393f288eca P e020e7ffe8ce423a8bdefec667161217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } }
I20240502 07:58:22.562391 6651 ts_tablet_manager.cc:1398] T 96dcb06388374514951f24393f288eca P e020e7ffe8ce423a8bdefec667161217: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:22.562929 6650 tablet_bootstrap.cc:654] T bc41794fb8c545149694aa84dde1cdb0 P d58f6d1ff8cf472f8a9edf3700e64bfb: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:22.563970 6650 tablet_bootstrap.cc:492] T bc41794fb8c545149694aa84dde1cdb0 P d58f6d1ff8cf472f8a9edf3700e64bfb: No bootstrap required, opened a new log
I20240502 07:58:22.564132 6650 ts_tablet_manager.cc:1367] T bc41794fb8c545149694aa84dde1cdb0 P d58f6d1ff8cf472f8a9edf3700e64bfb: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 07:58:22.564567 6650 raft_consensus.cc:348] T bc41794fb8c545149694aa84dde1cdb0 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } }
I20240502 07:58:22.564718 6650 raft_consensus.cc:374] T bc41794fb8c545149694aa84dde1cdb0 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:22.564770 6650 raft_consensus.cc:729] T bc41794fb8c545149694aa84dde1cdb0 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d58f6d1ff8cf472f8a9edf3700e64bfb, State: Initialized, Role: FOLLOWER
I20240502 07:58:22.564798 6655 tablet_bootstrap.cc:654] T 2a3919f144ed4a27b03e7c4a251499d4 P eda55a8566c447cfa749457825b70688: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:22.564925 6650 consensus_queue.cc:260] T bc41794fb8c545149694aa84dde1cdb0 P d58f6d1ff8cf472f8a9edf3700e64bfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } }
I20240502 07:58:22.565169 6650 ts_tablet_manager.cc:1398] T bc41794fb8c545149694aa84dde1cdb0 P d58f6d1ff8cf472f8a9edf3700e64bfb: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:22.565927 6655 tablet_bootstrap.cc:492] T 2a3919f144ed4a27b03e7c4a251499d4 P eda55a8566c447cfa749457825b70688: No bootstrap required, opened a new log
I20240502 07:58:22.566005 6655 ts_tablet_manager.cc:1367] T 2a3919f144ed4a27b03e7c4a251499d4 P eda55a8566c447cfa749457825b70688: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20240502 07:58:22.566355 6655 raft_consensus.cc:348] T 2a3919f144ed4a27b03e7c4a251499d4 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:22.566458 6655 raft_consensus.cc:374] T 2a3919f144ed4a27b03e7c4a251499d4 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:22.566488 6655 raft_consensus.cc:729] T 2a3919f144ed4a27b03e7c4a251499d4 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eda55a8566c447cfa749457825b70688, State: Initialized, Role: FOLLOWER
I20240502 07:58:22.566588 6655 consensus_queue.cc:260] T 2a3919f144ed4a27b03e7c4a251499d4 P eda55a8566c447cfa749457825b70688 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:22.566838 6655 ts_tablet_manager.cc:1398] T 2a3919f144ed4a27b03e7c4a251499d4 P eda55a8566c447cfa749457825b70688: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:22.579267 6551 raft_consensus.cc:482] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:22.579355 6551 raft_consensus.cc:504] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } }
I20240502 07:58:22.579865 6551 leader_election.cc:288] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eda55a8566c447cfa749457825b70688 (127.3.139.66:39925), d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68:44287), e020e7ffe8ce423a8bdefec667161217 (127.3.139.69:39399), d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70:41047)
I20240502 07:58:22.580202 5761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96dcb06388374514951f24393f288eca" candidate_uuid: "91e82003a656442189a329b29846a841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eda55a8566c447cfa749457825b70688" is_pre_election: true
I20240502 07:58:22.580324 5761 raft_consensus.cc:2454] T 96dcb06388374514951f24393f288eca P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91e82003a656442189a329b29846a841 in term 0.
I20240502 07:58:22.580451 6154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96dcb06388374514951f24393f288eca" candidate_uuid: "91e82003a656442189a329b29846a841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e020e7ffe8ce423a8bdefec667161217" is_pre_election: true
I20240502 07:58:22.580588 6154 raft_consensus.cc:2454] T 96dcb06388374514951f24393f288eca P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91e82003a656442189a329b29846a841 in term 0.
I20240502 07:58:22.581063 5564 leader_election.cc:302] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 91e82003a656442189a329b29846a841, e020e7ffe8ce423a8bdefec667161217, eda55a8566c447cfa749457825b70688; no voters:
I20240502 07:58:22.581257 6551 raft_consensus.cc:2781] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:22.581306 6551 raft_consensus.cc:482] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:22.581353 6551 raft_consensus.cc:3037] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:22.581611 6023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96dcb06388374514951f24393f288eca" candidate_uuid: "91e82003a656442189a329b29846a841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5bc7863cd294f77a932c4a40f98b79d" is_pre_election: true
I20240502 07:58:22.581744 6023 raft_consensus.cc:2454] T 96dcb06388374514951f24393f288eca P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91e82003a656442189a329b29846a841 in term 0.
I20240502 07:58:22.582322 6551 raft_consensus.cc:504] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } }
I20240502 07:58:22.582737 6551 leader_election.cc:288] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [CANDIDATE]: Term 1 election: Requested vote from peers eda55a8566c447cfa749457825b70688 (127.3.139.66:39925), d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68:44287), e020e7ffe8ce423a8bdefec667161217 (127.3.139.69:39399), d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70:41047)
I20240502 07:58:22.582974 5761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96dcb06388374514951f24393f288eca" candidate_uuid: "91e82003a656442189a329b29846a841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eda55a8566c447cfa749457825b70688"
I20240502 07:58:22.583029 6023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96dcb06388374514951f24393f288eca" candidate_uuid: "91e82003a656442189a329b29846a841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5bc7863cd294f77a932c4a40f98b79d"
I20240502 07:58:22.583101 5761 raft_consensus.cc:3037] T 96dcb06388374514951f24393f288eca P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:22.583134 6023 raft_consensus.cc:3037] T 96dcb06388374514951f24393f288eca P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:22.583972 5761 raft_consensus.cc:2454] T 96dcb06388374514951f24393f288eca P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91e82003a656442189a329b29846a841 in term 1.
I20240502 07:58:22.583972 6023 raft_consensus.cc:2454] T 96dcb06388374514951f24393f288eca P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91e82003a656442189a329b29846a841 in term 1.
I20240502 07:58:22.584270 6154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96dcb06388374514951f24393f288eca" candidate_uuid: "91e82003a656442189a329b29846a841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e020e7ffe8ce423a8bdefec667161217"
I20240502 07:58:22.584362 6154 raft_consensus.cc:3037] T 96dcb06388374514951f24393f288eca P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:22.584488 5564 leader_election.cc:302] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 91e82003a656442189a329b29846a841, d5bc7863cd294f77a932c4a40f98b79d, eda55a8566c447cfa749457825b70688; no voters:
I20240502 07:58:22.584611 6285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96dcb06388374514951f24393f288eca" candidate_uuid: "91e82003a656442189a329b29846a841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" is_pre_election: true
I20240502 07:58:22.584643 6551 raft_consensus.cc:2781] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:22.584719 6551 raft_consensus.cc:686] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [term 1 LEADER]: Becoming Leader. State: Replica: 91e82003a656442189a329b29846a841, State: Running, Role: LEADER
I20240502 07:58:22.584728 6285 raft_consensus.cc:2454] T 96dcb06388374514951f24393f288eca P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91e82003a656442189a329b29846a841 in term 0.
I20240502 07:58:22.584744 6284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96dcb06388374514951f24393f288eca" candidate_uuid: "91e82003a656442189a329b29846a841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb"
I20240502 07:58:22.584811 6284 raft_consensus.cc:3037] T 96dcb06388374514951f24393f288eca P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:22.584836 6551 consensus_queue.cc:237] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } }
I20240502 07:58:22.585310 6154 raft_consensus.cc:2454] T 96dcb06388374514951f24393f288eca P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91e82003a656442189a329b29846a841 in term 1.
I20240502 07:58:22.585759 6284 raft_consensus.cc:2454] T 96dcb06388374514951f24393f288eca P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91e82003a656442189a329b29846a841 in term 1.
I20240502 07:58:22.586668 5488 catalog_manager.cc:5484] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91e82003a656442189a329b29846a841 (127.3.139.65). New cstate: current_term: 1 leader_uuid: "91e82003a656442189a329b29846a841" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:22.599431 6552 raft_consensus.cc:482] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:22.599552 6552 raft_consensus.cc:504] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } }
I20240502 07:58:22.600910 6552 leader_election.cc:288] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68:44287), d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70:41047), eda55a8566c447cfa749457825b70688 (127.3.139.66:39925), e020e7ffe8ce423a8bdefec667161217 (127.3.139.69:39399)
I20240502 07:58:22.605104 6023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc41794fb8c545149694aa84dde1cdb0" candidate_uuid: "89bf5c0a6b2a4d949851089446f17b96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5bc7863cd294f77a932c4a40f98b79d" is_pre_election: true
I20240502 07:58:22.605218 6023 raft_consensus.cc:2454] T bc41794fb8c545149694aa84dde1cdb0 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89bf5c0a6b2a4d949851089446f17b96 in term 0.
I20240502 07:58:22.605278 6154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc41794fb8c545149694aa84dde1cdb0" candidate_uuid: "89bf5c0a6b2a4d949851089446f17b96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e020e7ffe8ce423a8bdefec667161217" is_pre_election: true
I20240502 07:58:22.605401 6154 raft_consensus.cc:2454] T bc41794fb8c545149694aa84dde1cdb0 P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89bf5c0a6b2a4d949851089446f17b96 in term 0.
I20240502 07:58:22.605449 6284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc41794fb8c545149694aa84dde1cdb0" candidate_uuid: "89bf5c0a6b2a4d949851089446f17b96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" is_pre_election: true
I20240502 07:58:22.605548 6284 raft_consensus.cc:2454] T bc41794fb8c545149694aa84dde1cdb0 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89bf5c0a6b2a4d949851089446f17b96 in term 0.
I20240502 07:58:22.606689 5761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc41794fb8c545149694aa84dde1cdb0" candidate_uuid: "89bf5c0a6b2a4d949851089446f17b96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eda55a8566c447cfa749457825b70688" is_pre_election: true
I20240502 07:58:22.605686 5827 leader_election.cc:302] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 89bf5c0a6b2a4d949851089446f17b96, d5bc7863cd294f77a932c4a40f98b79d, e020e7ffe8ce423a8bdefec667161217; no voters:
I20240502 07:58:22.606799 5761 raft_consensus.cc:2454] T bc41794fb8c545149694aa84dde1cdb0 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89bf5c0a6b2a4d949851089446f17b96 in term 0.
I20240502 07:58:22.606956 6552 raft_consensus.cc:2781] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:22.607009 6552 raft_consensus.cc:482] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:22.607062 6552 raft_consensus.cc:3037] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:22.607784 6552 raft_consensus.cc:504] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } }
I20240502 07:58:22.608171 6552 leader_election.cc:288] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96 [CANDIDATE]: Term 1 election: Requested vote from peers d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68:44287), d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70:41047), eda55a8566c447cfa749457825b70688 (127.3.139.66:39925), e020e7ffe8ce423a8bdefec667161217 (127.3.139.69:39399)
I20240502 07:58:22.608357 6023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc41794fb8c545149694aa84dde1cdb0" candidate_uuid: "89bf5c0a6b2a4d949851089446f17b96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5bc7863cd294f77a932c4a40f98b79d"
I20240502 07:58:22.608474 6023 raft_consensus.cc:3037] T bc41794fb8c545149694aa84dde1cdb0 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:22.608490 5761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc41794fb8c545149694aa84dde1cdb0" candidate_uuid: "89bf5c0a6b2a4d949851089446f17b96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eda55a8566c447cfa749457825b70688"
I20240502 07:58:22.608589 5761 raft_consensus.cc:3037] T bc41794fb8c545149694aa84dde1cdb0 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:22.608592 6154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc41794fb8c545149694aa84dde1cdb0" candidate_uuid: "89bf5c0a6b2a4d949851089446f17b96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e020e7ffe8ce423a8bdefec667161217"
I20240502 07:58:22.608592 6284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc41794fb8c545149694aa84dde1cdb0" candidate_uuid: "89bf5c0a6b2a4d949851089446f17b96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb"
I20240502 07:58:22.608671 6154 raft_consensus.cc:3037] T bc41794fb8c545149694aa84dde1cdb0 P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:22.608680 6284 raft_consensus.cc:3037] T bc41794fb8c545149694aa84dde1cdb0 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:22.609375 6023 raft_consensus.cc:2454] T bc41794fb8c545149694aa84dde1cdb0 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89bf5c0a6b2a4d949851089446f17b96 in term 1.
I20240502 07:58:22.609431 6284 raft_consensus.cc:2454] T bc41794fb8c545149694aa84dde1cdb0 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89bf5c0a6b2a4d949851089446f17b96 in term 1.
I20240502 07:58:22.609445 6154 raft_consensus.cc:2454] T bc41794fb8c545149694aa84dde1cdb0 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89bf5c0a6b2a4d949851089446f17b96 in term 1.
I20240502 07:58:22.609445 5761 raft_consensus.cc:2454] T bc41794fb8c545149694aa84dde1cdb0 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89bf5c0a6b2a4d949851089446f17b96 in term 1.
I20240502 07:58:22.609740 5826 leader_election.cc:302] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 89bf5c0a6b2a4d949851089446f17b96, d5bc7863cd294f77a932c4a40f98b79d, eda55a8566c447cfa749457825b70688; no voters:
I20240502 07:58:22.609946 6552 raft_consensus.cc:2781] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:22.610145 6552 raft_consensus.cc:686] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 LEADER]: Becoming Leader. State: Replica: 89bf5c0a6b2a4d949851089446f17b96, State: Running, Role: LEADER
I20240502 07:58:22.610407 6552 consensus_queue.cc:237] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } }
I20240502 07:58:22.612754 5488 catalog_manager.cc:5484] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96 reported cstate change: term changed from 0 to 1, leader changed from <none> to 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67). New cstate: current_term: 1 leader_uuid: "89bf5c0a6b2a4d949851089446f17b96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } health_report { overall_health: HEALTHY } } }
I20240502 07:58:22.632936 6549 raft_consensus.cc:482] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:22.633026 6549 raft_consensus.cc:504] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:22.634038 6549 leader_election.cc:288] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68:44287), eda55a8566c447cfa749457825b70688 (127.3.139.66:39925), d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70:41047), 91e82003a656442189a329b29846a841 (127.3.139.65:43429)
I20240502 07:58:22.638674 6284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a3919f144ed4a27b03e7c4a251499d4" candidate_uuid: "e020e7ffe8ce423a8bdefec667161217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" is_pre_election: true
I20240502 07:58:22.638788 6284 raft_consensus.cc:2454] T 2a3919f144ed4a27b03e7c4a251499d4 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e020e7ffe8ce423a8bdefec667161217 in term 0.
I20240502 07:58:22.638921 5761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a3919f144ed4a27b03e7c4a251499d4" candidate_uuid: "e020e7ffe8ce423a8bdefec667161217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eda55a8566c447cfa749457825b70688" is_pre_election: true
I20240502 07:58:22.638949 6023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a3919f144ed4a27b03e7c4a251499d4" candidate_uuid: "e020e7ffe8ce423a8bdefec667161217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5bc7863cd294f77a932c4a40f98b79d" is_pre_election: true
I20240502 07:58:22.639035 5761 raft_consensus.cc:2454] T 2a3919f144ed4a27b03e7c4a251499d4 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e020e7ffe8ce423a8bdefec667161217 in term 0.
I20240502 07:58:22.639093 6023 raft_consensus.cc:2454] T 2a3919f144ed4a27b03e7c4a251499d4 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e020e7ffe8ce423a8bdefec667161217 in term 0.
I20240502 07:58:22.639420 6089 leader_election.cc:302] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: d58f6d1ff8cf472f8a9edf3700e64bfb, d5bc7863cd294f77a932c4a40f98b79d, e020e7ffe8ce423a8bdefec667161217; no voters:
I20240502 07:58:22.639636 6549 raft_consensus.cc:2781] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:22.639704 6549 raft_consensus.cc:482] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:22.639752 6549 raft_consensus.cc:3037] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:22.639884 5628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a3919f144ed4a27b03e7c4a251499d4" candidate_uuid: "e020e7ffe8ce423a8bdefec667161217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e82003a656442189a329b29846a841" is_pre_election: true
I20240502 07:58:22.639995 5628 raft_consensus.cc:2454] T 2a3919f144ed4a27b03e7c4a251499d4 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e020e7ffe8ce423a8bdefec667161217 in term 0.
I20240502 07:58:22.640619 6549 raft_consensus.cc:504] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:22.641000 6549 leader_election.cc:288] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217 [CANDIDATE]: Term 1 election: Requested vote from peers d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68:44287), eda55a8566c447cfa749457825b70688 (127.3.139.66:39925), d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70:41047), 91e82003a656442189a329b29846a841 (127.3.139.65:43429)
I20240502 07:58:22.641258 6023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a3919f144ed4a27b03e7c4a251499d4" candidate_uuid: "e020e7ffe8ce423a8bdefec667161217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5bc7863cd294f77a932c4a40f98b79d"
I20240502 07:58:22.641253 6284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a3919f144ed4a27b03e7c4a251499d4" candidate_uuid: "e020e7ffe8ce423a8bdefec667161217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb"
I20240502 07:58:22.641357 6023 raft_consensus.cc:3037] T 2a3919f144ed4a27b03e7c4a251499d4 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:22.641367 6284 raft_consensus.cc:3037] T 2a3919f144ed4a27b03e7c4a251499d4 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:22.641983 6023 raft_consensus.cc:2454] T 2a3919f144ed4a27b03e7c4a251499d4 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e020e7ffe8ce423a8bdefec667161217 in term 1.
I20240502 07:58:22.641986 6284 raft_consensus.cc:2454] T 2a3919f144ed4a27b03e7c4a251499d4 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e020e7ffe8ce423a8bdefec667161217 in term 1.
I20240502 07:58:22.642212 5761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a3919f144ed4a27b03e7c4a251499d4" candidate_uuid: "e020e7ffe8ce423a8bdefec667161217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eda55a8566c447cfa749457825b70688"
I20240502 07:58:22.642300 5761 raft_consensus.cc:3037] T 2a3919f144ed4a27b03e7c4a251499d4 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:22.642948 5761 raft_consensus.cc:2454] T 2a3919f144ed4a27b03e7c4a251499d4 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e020e7ffe8ce423a8bdefec667161217 in term 1.
I20240502 07:58:22.643177 5628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a3919f144ed4a27b03e7c4a251499d4" candidate_uuid: "e020e7ffe8ce423a8bdefec667161217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e82003a656442189a329b29846a841"
I20240502 07:58:22.643213 6088 leader_election.cc:302] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: d5bc7863cd294f77a932c4a40f98b79d, e020e7ffe8ce423a8bdefec667161217, eda55a8566c447cfa749457825b70688; no voters:
I20240502 07:58:22.643276 5628 raft_consensus.cc:3037] T 2a3919f144ed4a27b03e7c4a251499d4 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:22.643371 6549 raft_consensus.cc:2781] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:22.643543 6549 raft_consensus.cc:686] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217 [term 1 LEADER]: Becoming Leader. State: Replica: e020e7ffe8ce423a8bdefec667161217, State: Running, Role: LEADER
I20240502 07:58:22.643833 6549 consensus_queue.cc:237] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } }
I20240502 07:58:22.644013 5628 raft_consensus.cc:2454] T 2a3919f144ed4a27b03e7c4a251499d4 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e020e7ffe8ce423a8bdefec667161217 in term 1.
I20240502 07:58:22.646041 5488 catalog_manager.cc:5484] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217 reported cstate change: term changed from 0 to 1, leader changed from <none> to e020e7ffe8ce423a8bdefec667161217 (127.3.139.69). New cstate: current_term: 1 leader_uuid: "e020e7ffe8ce423a8bdefec667161217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: UNKNOWN } } }
W20240502 07:58:22.780478 6815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:22.780694 6815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:22.783761 6815 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_3'
I20240502 07:58:22.873199 6284 raft_consensus.cc:1261] T 96dcb06388374514951f24393f288eca P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:22.873957 6023 raft_consensus.cc:1261] T 96dcb06388374514951f24393f288eca P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:22.874331 5761 raft_consensus.cc:1261] T bc41794fb8c545149694aa84dde1cdb0 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer 89bf5c0a6b2a4d949851089446f17b96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:22.874372 6022 raft_consensus.cc:1261] T bc41794fb8c545149694aa84dde1cdb0 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Refusing update from remote peer 89bf5c0a6b2a4d949851089446f17b96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:22.875401 5628 raft_consensus.cc:1261] T 2a3919f144ed4a27b03e7c4a251499d4 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer e020e7ffe8ce423a8bdefec667161217: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:22.875900 6813 consensus_queue.cc:1035] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:22.876315 5759 raft_consensus.cc:1261] T 2a3919f144ed4a27b03e7c4a251499d4 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer e020e7ffe8ce423a8bdefec667161217: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:22.876919 6154 raft_consensus.cc:1261] T bc41794fb8c545149694aa84dde1cdb0 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer 89bf5c0a6b2a4d949851089446f17b96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:22.876941 6285 raft_consensus.cc:1261] T 2a3919f144ed4a27b03e7c4a251499d4 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Refusing update from remote peer e020e7ffe8ce423a8bdefec667161217: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:22.877213 6153 raft_consensus.cc:1261] T 96dcb06388374514951f24393f288eca P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:22.877892 6843 consensus_queue.cc:1035] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:22.877944 6770 consensus_queue.cc:1035] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:22.878190 6022 raft_consensus.cc:1261] T 2a3919f144ed4a27b03e7c4a251499d4 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Refusing update from remote peer e020e7ffe8ce423a8bdefec667161217: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:22.878381 6807 consensus_queue.cc:1035] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:22.878557 6813 consensus_queue.cc:1035] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:22.880033 5761 raft_consensus.cc:1261] T 96dcb06388374514951f24393f288eca P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:22.880527 6770 consensus_queue.cc:1035] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:22.881179 6770 consensus_queue.cc:1035] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:22.881709 6284 raft_consensus.cc:1261] T bc41794fb8c545149694aa84dde1cdb0 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Refusing update from remote peer 89bf5c0a6b2a4d949851089446f17b96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:22.882527 6845 consensus_queue.cc:1035] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:22.882802 6549 consensus_queue.cc:1035] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:22.883697 6770 consensus_queue.cc:1035] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:22.885690 6807 consensus_queue.cc:1035] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:22.885982 6807 consensus_queue.cc:1035] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 155.103 ms
time per row: 0.0775514 ms
I20240502 07:58:22.978366 5481 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40652:
name: "rebalance_test_table_4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 5
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:58:22.983728 5610 tablet_service.cc:1467] Processing CreateTablet for tablet eede21c622d8435ba1efa473ef97b1d1 (DEFAULT_TABLE table=rebalance_test_table_4 [id=5c1e49d6edf446d5a06063e568da2a0a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:22.983788 5872 tablet_service.cc:1467] Processing CreateTablet for tablet eede21c622d8435ba1efa473ef97b1d1 (DEFAULT_TABLE table=rebalance_test_table_4 [id=5c1e49d6edf446d5a06063e568da2a0a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:22.984040 5872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eede21c622d8435ba1efa473ef97b1d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:22.984047 5610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eede21c622d8435ba1efa473ef97b1d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:22.986068 6529 tablet_bootstrap.cc:492] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841: Bootstrap starting.
I20240502 07:58:22.986071 6542 tablet_bootstrap.cc:492] T eede21c622d8435ba1efa473ef97b1d1 P 89bf5c0a6b2a4d949851089446f17b96: Bootstrap starting.
I20240502 07:58:22.987167 6529 tablet_bootstrap.cc:654] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:22.987200 6132 tablet_service.cc:1467] Processing CreateTablet for tablet eede21c622d8435ba1efa473ef97b1d1 (DEFAULT_TABLE table=rebalance_test_table_4 [id=5c1e49d6edf446d5a06063e568da2a0a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:22.987483 6132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eede21c622d8435ba1efa473ef97b1d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:22.988193 6529 tablet_bootstrap.cc:492] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841: No bootstrap required, opened a new log
I20240502 07:58:22.988277 6529 ts_tablet_manager.cc:1367] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 07:58:22.988683 6529 raft_consensus.cc:348] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } }
I20240502 07:58:22.988818 6529 raft_consensus.cc:374] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:22.988862 6529 raft_consensus.cc:729] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91e82003a656442189a329b29846a841, State: Initialized, Role: FOLLOWER
I20240502 07:58:22.988977 6529 consensus_queue.cc:260] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } }
I20240502 07:58:22.989238 6529 ts_tablet_manager.cc:1398] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:22.989476 6651 tablet_bootstrap.cc:492] T eede21c622d8435ba1efa473ef97b1d1 P e020e7ffe8ce423a8bdefec667161217: Bootstrap starting.
I20240502 07:58:22.990389 6651 tablet_bootstrap.cc:654] T eede21c622d8435ba1efa473ef97b1d1 P e020e7ffe8ce423a8bdefec667161217: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:22.991637 6651 tablet_bootstrap.cc:492] T eede21c622d8435ba1efa473ef97b1d1 P e020e7ffe8ce423a8bdefec667161217: No bootstrap required, opened a new log
I20240502 07:58:22.991588 6263 tablet_service.cc:1467] Processing CreateTablet for tablet 7f3d5d24bc164883adb1a11552500ce6 (DEFAULT_TABLE table=rebalance_test_table_4 [id=5c1e49d6edf446d5a06063e568da2a0a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:22.991588 6265 tablet_service.cc:1467] Processing CreateTablet for tablet 07d53332bdd34233b3653756770dc9cf (DEFAULT_TABLE table=rebalance_test_table_4 [id=5c1e49d6edf446d5a06063e568da2a0a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:22.991724 6651 ts_tablet_manager.cc:1367] T eede21c622d8435ba1efa473ef97b1d1 P e020e7ffe8ce423a8bdefec667161217: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:22.991894 6263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f3d5d24bc164883adb1a11552500ce6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:22.992132 6651 raft_consensus.cc:348] T eede21c622d8435ba1efa473ef97b1d1 P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } }
I20240502 07:58:22.992235 6651 raft_consensus.cc:374] T eede21c622d8435ba1efa473ef97b1d1 P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:22.992256 6265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07d53332bdd34233b3653756770dc9cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:22.992267 6651 raft_consensus.cc:729] T eede21c622d8435ba1efa473ef97b1d1 P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e020e7ffe8ce423a8bdefec667161217, State: Initialized, Role: FOLLOWER
I20240502 07:58:22.992393 6651 consensus_queue.cc:260] T eede21c622d8435ba1efa473ef97b1d1 P e020e7ffe8ce423a8bdefec667161217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } }
I20240502 07:58:22.992661 6651 ts_tablet_manager.cc:1398] T eede21c622d8435ba1efa473ef97b1d1 P e020e7ffe8ce423a8bdefec667161217: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:22.992736 6542 tablet_bootstrap.cc:654] T eede21c622d8435ba1efa473ef97b1d1 P 89bf5c0a6b2a4d949851089446f17b96: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:22.994441 6650 tablet_bootstrap.cc:492] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb: Bootstrap starting.
I20240502 07:58:22.994699 5739 tablet_service.cc:1467] Processing CreateTablet for tablet eede21c622d8435ba1efa473ef97b1d1 (DEFAULT_TABLE table=rebalance_test_table_4 [id=5c1e49d6edf446d5a06063e568da2a0a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:22.995237 5739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eede21c622d8435ba1efa473ef97b1d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:22.995378 6650 tablet_bootstrap.cc:654] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:22.996124 6542 tablet_bootstrap.cc:492] T eede21c622d8435ba1efa473ef97b1d1 P 89bf5c0a6b2a4d949851089446f17b96: No bootstrap required, opened a new log
I20240502 07:58:22.996208 6542 ts_tablet_manager.cc:1367] T eede21c622d8435ba1efa473ef97b1d1 P 89bf5c0a6b2a4d949851089446f17b96: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.001s
I20240502 07:58:22.996553 6001 tablet_service.cc:1467] Processing CreateTablet for tablet eede21c622d8435ba1efa473ef97b1d1 (DEFAULT_TABLE table=rebalance_test_table_4 [id=5c1e49d6edf446d5a06063e568da2a0a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:22.996636 6542 raft_consensus.cc:348] T eede21c622d8435ba1efa473ef97b1d1 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } }
I20240502 07:58:22.996730 6542 raft_consensus.cc:374] T eede21c622d8435ba1efa473ef97b1d1 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:22.996759 6542 raft_consensus.cc:729] T eede21c622d8435ba1efa473ef97b1d1 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89bf5c0a6b2a4d949851089446f17b96, State: Initialized, Role: FOLLOWER
I20240502 07:58:22.996838 6650 tablet_bootstrap.cc:492] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb: No bootstrap required, opened a new log
I20240502 07:58:22.996807 6001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eede21c622d8435ba1efa473ef97b1d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:22.996801 6002 tablet_service.cc:1467] Processing CreateTablet for tablet 7f3d5d24bc164883adb1a11552500ce6 (DEFAULT_TABLE table=rebalance_test_table_4 [id=5c1e49d6edf446d5a06063e568da2a0a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:22.996905 6650 ts_tablet_manager.cc:1367] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:58:22.996870 6542 consensus_queue.cc:260] T eede21c622d8435ba1efa473ef97b1d1 P 89bf5c0a6b2a4d949851089446f17b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } }
I20240502 07:58:22.997052 6002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f3d5d24bc164883adb1a11552500ce6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:22.997129 6542 ts_tablet_manager.cc:1398] T eede21c622d8435ba1efa473ef97b1d1 P 89bf5c0a6b2a4d949851089446f17b96: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:22.997658 6650 raft_consensus.cc:348] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } }
I20240502 07:58:22.997756 6650 raft_consensus.cc:374] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:22.997784 6650 raft_consensus.cc:729] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d58f6d1ff8cf472f8a9edf3700e64bfb, State: Initialized, Role: FOLLOWER
I20240502 07:58:22.997747 5870 tablet_service.cc:1467] Processing CreateTablet for tablet 07d53332bdd34233b3653756770dc9cf (DEFAULT_TABLE table=rebalance_test_table_4 [id=5c1e49d6edf446d5a06063e568da2a0a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:22.997892 6650 consensus_queue.cc:260] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } }
I20240502 07:58:22.997978 5870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07d53332bdd34233b3653756770dc9cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:22.998178 6650 ts_tablet_manager.cc:1398] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:22.999783 5872 tablet_service.cc:1467] Processing CreateTablet for tablet 7f3d5d24bc164883adb1a11552500ce6 (DEFAULT_TABLE table=rebalance_test_table_4 [id=5c1e49d6edf446d5a06063e568da2a0a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:22.999886 6542 tablet_bootstrap.cc:492] T 07d53332bdd34233b3653756770dc9cf P 89bf5c0a6b2a4d949851089446f17b96: Bootstrap starting.
I20240502 07:58:23.000425 5740 tablet_service.cc:1467] Processing CreateTablet for tablet 7f3d5d24bc164883adb1a11552500ce6 (DEFAULT_TABLE table=rebalance_test_table_4 [id=5c1e49d6edf446d5a06063e568da2a0a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:23.000487 5872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f3d5d24bc164883adb1a11552500ce6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:23.000507 5741 tablet_service.cc:1467] Processing CreateTablet for tablet 07d53332bdd34233b3653756770dc9cf (DEFAULT_TABLE table=rebalance_test_table_4 [id=5c1e49d6edf446d5a06063e568da2a0a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:23.000651 5740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f3d5d24bc164883adb1a11552500ce6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:23.000813 5741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07d53332bdd34233b3653756770dc9cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:23.002426 6542 tablet_bootstrap.cc:654] T 07d53332bdd34233b3653756770dc9cf P 89bf5c0a6b2a4d949851089446f17b96: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:23.003535 6542 tablet_bootstrap.cc:492] T 07d53332bdd34233b3653756770dc9cf P 89bf5c0a6b2a4d949851089446f17b96: No bootstrap required, opened a new log
I20240502 07:58:23.003605 6542 ts_tablet_manager.cc:1367] T 07d53332bdd34233b3653756770dc9cf P 89bf5c0a6b2a4d949851089446f17b96: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20240502 07:58:23.003576 5610 tablet_service.cc:1467] Processing CreateTablet for tablet 7f3d5d24bc164883adb1a11552500ce6 (DEFAULT_TABLE table=rebalance_test_table_4 [id=5c1e49d6edf446d5a06063e568da2a0a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:23.003612 5608 tablet_service.cc:1467] Processing CreateTablet for tablet 07d53332bdd34233b3653756770dc9cf (DEFAULT_TABLE table=rebalance_test_table_4 [id=5c1e49d6edf446d5a06063e568da2a0a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:23.003824 5610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f3d5d24bc164883adb1a11552500ce6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:23.003958 5608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07d53332bdd34233b3653756770dc9cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:23.004004 6542 raft_consensus.cc:348] T 07d53332bdd34233b3653756770dc9cf P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } }
I20240502 07:58:23.004089 6542 raft_consensus.cc:374] T 07d53332bdd34233b3653756770dc9cf P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:23.004117 6542 raft_consensus.cc:729] T 07d53332bdd34233b3653756770dc9cf P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89bf5c0a6b2a4d949851089446f17b96, State: Initialized, Role: FOLLOWER
I20240502 07:58:23.004253 6542 consensus_queue.cc:260] T 07d53332bdd34233b3653756770dc9cf P 89bf5c0a6b2a4d949851089446f17b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } }
I20240502 07:58:23.004565 6542 ts_tablet_manager.cc:1398] T 07d53332bdd34233b3653756770dc9cf P 89bf5c0a6b2a4d949851089446f17b96: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:23.004706 6542 tablet_bootstrap.cc:492] T 7f3d5d24bc164883adb1a11552500ce6 P 89bf5c0a6b2a4d949851089446f17b96: Bootstrap starting.
I20240502 07:58:23.005023 6648 tablet_bootstrap.cc:492] T eede21c622d8435ba1efa473ef97b1d1 P d5bc7863cd294f77a932c4a40f98b79d: Bootstrap starting.
I20240502 07:58:23.005735 6648 tablet_bootstrap.cc:654] T eede21c622d8435ba1efa473ef97b1d1 P d5bc7863cd294f77a932c4a40f98b79d: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:23.005754 6529 tablet_bootstrap.cc:492] T 7f3d5d24bc164883adb1a11552500ce6 P 91e82003a656442189a329b29846a841: Bootstrap starting.
I20240502 07:58:23.006603 6529 tablet_bootstrap.cc:654] T 7f3d5d24bc164883adb1a11552500ce6 P 91e82003a656442189a329b29846a841: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:23.007687 6542 tablet_bootstrap.cc:654] T 7f3d5d24bc164883adb1a11552500ce6 P 89bf5c0a6b2a4d949851089446f17b96: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:23.008375 6770 raft_consensus.cc:482] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:23.008456 6770 raft_consensus.cc:504] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } }
I20240502 07:58:23.008915 6770 leader_election.cc:288] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67:37141), eda55a8566c447cfa749457825b70688 (127.3.139.66:39925), d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68:44287), e020e7ffe8ce423a8bdefec667161217 (127.3.139.69:39399)
I20240502 07:58:23.009158 6648 tablet_bootstrap.cc:492] T eede21c622d8435ba1efa473ef97b1d1 P d5bc7863cd294f77a932c4a40f98b79d: No bootstrap required, opened a new log
I20240502 07:58:23.009229 6648 ts_tablet_manager.cc:1367] T eede21c622d8435ba1efa473ef97b1d1 P d5bc7863cd294f77a932c4a40f98b79d: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 07:58:23.009472 5892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eede21c622d8435ba1efa473ef97b1d1" candidate_uuid: "91e82003a656442189a329b29846a841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89bf5c0a6b2a4d949851089446f17b96" is_pre_election: true
I20240502 07:58:23.009578 5892 raft_consensus.cc:2454] T eede21c622d8435ba1efa473ef97b1d1 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91e82003a656442189a329b29846a841 in term 0.
I20240502 07:58:23.009618 6648 raft_consensus.cc:348] T eede21c622d8435ba1efa473ef97b1d1 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } }
I20240502 07:58:23.009717 6648 raft_consensus.cc:374] T eede21c622d8435ba1efa473ef97b1d1 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:23.009747 6648 raft_consensus.cc:729] T eede21c622d8435ba1efa473ef97b1d1 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5bc7863cd294f77a932c4a40f98b79d, State: Initialized, Role: FOLLOWER
I20240502 07:58:23.009848 6648 consensus_queue.cc:260] T eede21c622d8435ba1efa473ef97b1d1 P d5bc7863cd294f77a932c4a40f98b79d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } }
I20240502 07:58:23.010102 6648 ts_tablet_manager.cc:1398] T eede21c622d8435ba1efa473ef97b1d1 P d5bc7863cd294f77a932c4a40f98b79d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20240502 07:58:23.010289 5564 leader_election.cc:341] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eda55a8566c447cfa749457825b70688 (127.3.139.66:39925): Not found: Tablet not found: eede21c622d8435ba1efa473ef97b1d1
I20240502 07:58:23.010469 6154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eede21c622d8435ba1efa473ef97b1d1" candidate_uuid: "91e82003a656442189a329b29846a841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e020e7ffe8ce423a8bdefec667161217" is_pre_election: true
I20240502 07:58:23.010607 6154 raft_consensus.cc:2454] T eede21c622d8435ba1efa473ef97b1d1 P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91e82003a656442189a329b29846a841 in term 0.
I20240502 07:58:23.011197 6132 tablet_service.cc:1467] Processing CreateTablet for tablet 07d53332bdd34233b3653756770dc9cf (DEFAULT_TABLE table=rebalance_test_table_4 [id=5c1e49d6edf446d5a06063e568da2a0a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:23.011456 6132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07d53332bdd34233b3653756770dc9cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:23.011463 6023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eede21c622d8435ba1efa473ef97b1d1" candidate_uuid: "91e82003a656442189a329b29846a841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5bc7863cd294f77a932c4a40f98b79d" is_pre_election: true
I20240502 07:58:23.011551 6023 raft_consensus.cc:2454] T eede21c622d8435ba1efa473ef97b1d1 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91e82003a656442189a329b29846a841 in term 0.
I20240502 07:58:23.013316 6651 tablet_bootstrap.cc:492] T 07d53332bdd34233b3653756770dc9cf P e020e7ffe8ce423a8bdefec667161217: Bootstrap starting.
I20240502 07:58:23.013348 6650 tablet_bootstrap.cc:492] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb: Bootstrap starting.
I20240502 07:58:23.014137 6650 tablet_bootstrap.cc:654] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:23.014374 6651 tablet_bootstrap.cc:654] T 07d53332bdd34233b3653756770dc9cf P e020e7ffe8ce423a8bdefec667161217: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:23.015190 6650 tablet_bootstrap.cc:492] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb: No bootstrap required, opened a new log
I20240502 07:58:23.015264 6650 ts_tablet_manager.cc:1367] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:23.015439 6651 tablet_bootstrap.cc:492] T 07d53332bdd34233b3653756770dc9cf P e020e7ffe8ce423a8bdefec667161217: No bootstrap required, opened a new log
I20240502 07:58:23.015514 6651 ts_tablet_manager.cc:1367] T 07d53332bdd34233b3653756770dc9cf P e020e7ffe8ce423a8bdefec667161217: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:23.015692 6650 raft_consensus.cc:348] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } }
I20240502 07:58:23.015790 6650 raft_consensus.cc:374] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:23.015826 6650 raft_consensus.cc:729] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d58f6d1ff8cf472f8a9edf3700e64bfb, State: Initialized, Role: FOLLOWER
I20240502 07:58:23.015957 6650 consensus_queue.cc:260] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } }
I20240502 07:58:23.016019 5565 leader_election.cc:302] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 5 voters: 3 yes votes; 1 no votes. yes voters: 89bf5c0a6b2a4d949851089446f17b96, 91e82003a656442189a329b29846a841, e020e7ffe8ce423a8bdefec667161217; no voters: eda55a8566c447cfa749457825b70688
I20240502 07:58:23.016346 6650 ts_tablet_manager.cc:1398] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:23.016335 6770 raft_consensus.cc:2781] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:23.016386 6770 raft_consensus.cc:482] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:23.016422 6770 raft_consensus.cc:3037] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:23.016949 6651 raft_consensus.cc:348] T 07d53332bdd34233b3653756770dc9cf P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } }
I20240502 07:58:23.017050 6651 raft_consensus.cc:374] T 07d53332bdd34233b3653756770dc9cf P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:23.017081 6651 raft_consensus.cc:729] T 07d53332bdd34233b3653756770dc9cf P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e020e7ffe8ce423a8bdefec667161217, State: Initialized, Role: FOLLOWER
I20240502 07:58:23.017200 6651 consensus_queue.cc:260] T 07d53332bdd34233b3653756770dc9cf P e020e7ffe8ce423a8bdefec667161217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } }
I20240502 07:58:23.017210 6770 raft_consensus.cc:504] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } }
I20240502 07:58:23.017633 6770 leader_election.cc:288] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841 [CANDIDATE]: Term 1 election: Requested vote from peers 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67:37141), eda55a8566c447cfa749457825b70688 (127.3.139.66:39925), d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68:44287), e020e7ffe8ce423a8bdefec667161217 (127.3.139.69:39399)
I20240502 07:58:23.017971 6023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eede21c622d8435ba1efa473ef97b1d1" candidate_uuid: "91e82003a656442189a329b29846a841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5bc7863cd294f77a932c4a40f98b79d"
I20240502 07:58:23.017976 5892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eede21c622d8435ba1efa473ef97b1d1" candidate_uuid: "91e82003a656442189a329b29846a841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89bf5c0a6b2a4d949851089446f17b96"
I20240502 07:58:23.018060 6023 raft_consensus.cc:3037] T eede21c622d8435ba1efa473ef97b1d1 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:23.018075 5892 raft_consensus.cc:3037] T eede21c622d8435ba1efa473ef97b1d1 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:23.018625 6023 raft_consensus.cc:2454] T eede21c622d8435ba1efa473ef97b1d1 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91e82003a656442189a329b29846a841 in term 1.
W20240502 07:58:23.018957 5564 leader_election.cc:341] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eda55a8566c447cfa749457825b70688 (127.3.139.66:39925): Not found: Tablet not found: eede21c622d8435ba1efa473ef97b1d1
I20240502 07:58:23.018939 6154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eede21c622d8435ba1efa473ef97b1d1" candidate_uuid: "91e82003a656442189a329b29846a841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e020e7ffe8ce423a8bdefec667161217"
I20240502 07:58:23.019035 6154 raft_consensus.cc:3037] T eede21c622d8435ba1efa473ef97b1d1 P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:23.019217 6651 ts_tablet_manager.cc:1398] T 07d53332bdd34233b3653756770dc9cf P e020e7ffe8ce423a8bdefec667161217: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20240502 07:58:23.019863 6154 raft_consensus.cc:2454] T eede21c622d8435ba1efa473ef97b1d1 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91e82003a656442189a329b29846a841 in term 1.
I20240502 07:58:23.020995 6655 tablet_bootstrap.cc:492] T eede21c622d8435ba1efa473ef97b1d1 P eda55a8566c447cfa749457825b70688: Bootstrap starting.
I20240502 07:58:23.021854 6655 tablet_bootstrap.cc:654] T eede21c622d8435ba1efa473ef97b1d1 P eda55a8566c447cfa749457825b70688: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:23.022079 5565 leader_election.cc:302] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 5 voters: 3 yes votes; 1 no votes. yes voters: 91e82003a656442189a329b29846a841, d5bc7863cd294f77a932c4a40f98b79d, e020e7ffe8ce423a8bdefec667161217; no voters: eda55a8566c447cfa749457825b70688
I20240502 07:58:23.022291 6648 tablet_bootstrap.cc:492] T 7f3d5d24bc164883adb1a11552500ce6 P d5bc7863cd294f77a932c4a40f98b79d: Bootstrap starting.
I20240502 07:58:23.022979 6770 raft_consensus.cc:2781] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:23.023072 6770 raft_consensus.cc:686] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841 [term 1 LEADER]: Becoming Leader. State: Replica: 91e82003a656442189a329b29846a841, State: Running, Role: LEADER
I20240502 07:58:23.023162 6648 tablet_bootstrap.cc:654] T 7f3d5d24bc164883adb1a11552500ce6 P d5bc7863cd294f77a932c4a40f98b79d: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:23.023195 6770 consensus_queue.cc:237] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } }
I20240502 07:58:23.024124 6648 tablet_bootstrap.cc:492] T 7f3d5d24bc164883adb1a11552500ce6 P d5bc7863cd294f77a932c4a40f98b79d: No bootstrap required, opened a new log
I20240502 07:58:23.024204 6648 ts_tablet_manager.cc:1367] T 7f3d5d24bc164883adb1a11552500ce6 P d5bc7863cd294f77a932c4a40f98b79d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 07:58:23.024982 6648 raft_consensus.cc:348] T 7f3d5d24bc164883adb1a11552500ce6 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } }
I20240502 07:58:23.025084 6648 raft_consensus.cc:374] T 7f3d5d24bc164883adb1a11552500ce6 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:23.025113 6648 raft_consensus.cc:729] T 7f3d5d24bc164883adb1a11552500ce6 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5bc7863cd294f77a932c4a40f98b79d, State: Initialized, Role: FOLLOWER
I20240502 07:58:23.025255 6655 tablet_bootstrap.cc:492] T eede21c622d8435ba1efa473ef97b1d1 P eda55a8566c447cfa749457825b70688: No bootstrap required, opened a new log
I20240502 07:58:23.025226 6648 consensus_queue.cc:260] T 7f3d5d24bc164883adb1a11552500ce6 P d5bc7863cd294f77a932c4a40f98b79d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } }
I20240502 07:58:23.025319 6655 ts_tablet_manager.cc:1367] T eede21c622d8435ba1efa473ef97b1d1 P eda55a8566c447cfa749457825b70688: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240502 07:58:23.025483 6648 ts_tablet_manager.cc:1398] T 7f3d5d24bc164883adb1a11552500ce6 P d5bc7863cd294f77a932c4a40f98b79d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:23.025725 6655 raft_consensus.cc:348] T eede21c622d8435ba1efa473ef97b1d1 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } }
I20240502 07:58:23.025828 6655 raft_consensus.cc:374] T eede21c622d8435ba1efa473ef97b1d1 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:23.025861 6655 raft_consensus.cc:729] T eede21c622d8435ba1efa473ef97b1d1 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eda55a8566c447cfa749457825b70688, State: Initialized, Role: FOLLOWER
I20240502 07:58:23.025979 6655 consensus_queue.cc:260] T eede21c622d8435ba1efa473ef97b1d1 P eda55a8566c447cfa749457825b70688 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } }
I20240502 07:58:23.026259 6655 ts_tablet_manager.cc:1398] T eede21c622d8435ba1efa473ef97b1d1 P eda55a8566c447cfa749457825b70688: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:23.026460 6655 tablet_bootstrap.cc:492] T 07d53332bdd34233b3653756770dc9cf P eda55a8566c447cfa749457825b70688: Bootstrap starting.
I20240502 07:58:23.026593 5488 catalog_manager.cc:5484] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91e82003a656442189a329b29846a841 (127.3.139.65). New cstate: current_term: 1 leader_uuid: "91e82003a656442189a329b29846a841" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:23.027278 6655 tablet_bootstrap.cc:654] T 07d53332bdd34233b3653756770dc9cf P eda55a8566c447cfa749457825b70688: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:23.028700 6655 tablet_bootstrap.cc:492] T 07d53332bdd34233b3653756770dc9cf P eda55a8566c447cfa749457825b70688: No bootstrap required, opened a new log
I20240502 07:58:23.028831 6655 ts_tablet_manager.cc:1367] T 07d53332bdd34233b3653756770dc9cf P eda55a8566c447cfa749457825b70688: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:23.029253 6655 raft_consensus.cc:348] T 07d53332bdd34233b3653756770dc9cf P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } }
I20240502 07:58:23.029342 6655 raft_consensus.cc:374] T 07d53332bdd34233b3653756770dc9cf P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:23.029371 6655 raft_consensus.cc:729] T 07d53332bdd34233b3653756770dc9cf P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eda55a8566c447cfa749457825b70688, State: Initialized, Role: FOLLOWER
I20240502 07:58:23.029482 6655 consensus_queue.cc:260] T 07d53332bdd34233b3653756770dc9cf P eda55a8566c447cfa749457825b70688 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } }
I20240502 07:58:23.029721 6655 ts_tablet_manager.cc:1398] T 07d53332bdd34233b3653756770dc9cf P eda55a8566c447cfa749457825b70688: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:23.029865 6655 tablet_bootstrap.cc:492] T 7f3d5d24bc164883adb1a11552500ce6 P eda55a8566c447cfa749457825b70688: Bootstrap starting.
I20240502 07:58:23.030664 6655 tablet_bootstrap.cc:654] T 7f3d5d24bc164883adb1a11552500ce6 P eda55a8566c447cfa749457825b70688: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:23.031378 6542 tablet_bootstrap.cc:492] T 7f3d5d24bc164883adb1a11552500ce6 P 89bf5c0a6b2a4d949851089446f17b96: No bootstrap required, opened a new log
I20240502 07:58:23.031464 6542 ts_tablet_manager.cc:1367] T 7f3d5d24bc164883adb1a11552500ce6 P 89bf5c0a6b2a4d949851089446f17b96: Time spent bootstrapping tablet: real 0.027s user 0.002s sys 0.000s
I20240502 07:58:23.031832 6542 raft_consensus.cc:348] T 7f3d5d24bc164883adb1a11552500ce6 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } }
I20240502 07:58:23.031939 6542 raft_consensus.cc:374] T 7f3d5d24bc164883adb1a11552500ce6 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:23.031965 6542 raft_consensus.cc:729] T 7f3d5d24bc164883adb1a11552500ce6 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89bf5c0a6b2a4d949851089446f17b96, State: Initialized, Role: FOLLOWER
I20240502 07:58:23.032068 6542 consensus_queue.cc:260] T 7f3d5d24bc164883adb1a11552500ce6 P 89bf5c0a6b2a4d949851089446f17b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } }
I20240502 07:58:23.032292 6542 ts_tablet_manager.cc:1398] T 7f3d5d24bc164883adb1a11552500ce6 P 89bf5c0a6b2a4d949851089446f17b96: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:23.033109 5892 raft_consensus.cc:2454] T eede21c622d8435ba1efa473ef97b1d1 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91e82003a656442189a329b29846a841 in term 1.
I20240502 07:58:23.033831 6655 tablet_bootstrap.cc:492] T 7f3d5d24bc164883adb1a11552500ce6 P eda55a8566c447cfa749457825b70688: No bootstrap required, opened a new log
I20240502 07:58:23.033951 6655 ts_tablet_manager.cc:1367] T 7f3d5d24bc164883adb1a11552500ce6 P eda55a8566c447cfa749457825b70688: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240502 07:58:23.034435 6529 tablet_bootstrap.cc:492] T 7f3d5d24bc164883adb1a11552500ce6 P 91e82003a656442189a329b29846a841: No bootstrap required, opened a new log
I20240502 07:58:23.034508 6529 ts_tablet_manager.cc:1367] T 7f3d5d24bc164883adb1a11552500ce6 P 91e82003a656442189a329b29846a841: Time spent bootstrapping tablet: real 0.029s user 0.000s sys 0.002s
I20240502 07:58:23.034605 6655 raft_consensus.cc:348] T 7f3d5d24bc164883adb1a11552500ce6 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } }
I20240502 07:58:23.034698 6655 raft_consensus.cc:374] T 7f3d5d24bc164883adb1a11552500ce6 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:23.034725 6655 raft_consensus.cc:729] T 7f3d5d24bc164883adb1a11552500ce6 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eda55a8566c447cfa749457825b70688, State: Initialized, Role: FOLLOWER
I20240502 07:58:23.034828 6655 consensus_queue.cc:260] T 7f3d5d24bc164883adb1a11552500ce6 P eda55a8566c447cfa749457825b70688 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } }
I20240502 07:58:23.034883 6529 raft_consensus.cc:348] T 7f3d5d24bc164883adb1a11552500ce6 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } }
I20240502 07:58:23.034982 6529 raft_consensus.cc:374] T 7f3d5d24bc164883adb1a11552500ce6 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:23.035006 6529 raft_consensus.cc:729] T 7f3d5d24bc164883adb1a11552500ce6 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91e82003a656442189a329b29846a841, State: Initialized, Role: FOLLOWER
I20240502 07:58:23.035079 6655 ts_tablet_manager.cc:1398] T 7f3d5d24bc164883adb1a11552500ce6 P eda55a8566c447cfa749457825b70688: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:23.035120 6529 consensus_queue.cc:260] T 7f3d5d24bc164883adb1a11552500ce6 P 91e82003a656442189a329b29846a841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } }
I20240502 07:58:23.035346 6529 ts_tablet_manager.cc:1398] T 7f3d5d24bc164883adb1a11552500ce6 P 91e82003a656442189a329b29846a841: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:23.035476 6529 tablet_bootstrap.cc:492] T 07d53332bdd34233b3653756770dc9cf P 91e82003a656442189a329b29846a841: Bootstrap starting.
I20240502 07:58:23.036399 6529 tablet_bootstrap.cc:654] T 07d53332bdd34233b3653756770dc9cf P 91e82003a656442189a329b29846a841: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:23.039963 6529 tablet_bootstrap.cc:492] T 07d53332bdd34233b3653756770dc9cf P 91e82003a656442189a329b29846a841: No bootstrap required, opened a new log
I20240502 07:58:23.040042 6529 ts_tablet_manager.cc:1367] T 07d53332bdd34233b3653756770dc9cf P 91e82003a656442189a329b29846a841: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 07:58:23.040426 6529 raft_consensus.cc:348] T 07d53332bdd34233b3653756770dc9cf P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } }
I20240502 07:58:23.040532 6529 raft_consensus.cc:374] T 07d53332bdd34233b3653756770dc9cf P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:23.040560 6529 raft_consensus.cc:729] T 07d53332bdd34233b3653756770dc9cf P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91e82003a656442189a329b29846a841, State: Initialized, Role: FOLLOWER
I20240502 07:58:23.040675 6529 consensus_queue.cc:260] T 07d53332bdd34233b3653756770dc9cf P 91e82003a656442189a329b29846a841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } }
I20240502 07:58:23.041591 6529 ts_tablet_manager.cc:1398] T 07d53332bdd34233b3653756770dc9cf P 91e82003a656442189a329b29846a841: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:23.192991 6884 raft_consensus.cc:482] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:23.193090 6884 raft_consensus.cc:504] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } }
I20240502 07:58:23.193542 6884 leader_election.cc:288] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e020e7ffe8ce423a8bdefec667161217 (127.3.139.69:39399), 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67:37141), 91e82003a656442189a329b29846a841 (127.3.139.65:43429), eda55a8566c447cfa749457825b70688 (127.3.139.66:39925)
I20240502 07:58:23.193810 6154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07d53332bdd34233b3653756770dc9cf" candidate_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e020e7ffe8ce423a8bdefec667161217" is_pre_election: true
I20240502 07:58:23.193822 5892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07d53332bdd34233b3653756770dc9cf" candidate_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89bf5c0a6b2a4d949851089446f17b96" is_pre_election: true
I20240502 07:58:23.193962 6154 raft_consensus.cc:2454] T 07d53332bdd34233b3653756770dc9cf P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d58f6d1ff8cf472f8a9edf3700e64bfb in term 0.
I20240502 07:58:23.193990 5892 raft_consensus.cc:2454] T 07d53332bdd34233b3653756770dc9cf P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d58f6d1ff8cf472f8a9edf3700e64bfb in term 0.
I20240502 07:58:23.194092 5628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07d53332bdd34233b3653756770dc9cf" candidate_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e82003a656442189a329b29846a841" is_pre_election: true
I20240502 07:58:23.194279 5628 raft_consensus.cc:2454] T 07d53332bdd34233b3653756770dc9cf P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d58f6d1ff8cf472f8a9edf3700e64bfb in term 0.
I20240502 07:58:23.194491 6219 leader_election.cc:302] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 89bf5c0a6b2a4d949851089446f17b96, d58f6d1ff8cf472f8a9edf3700e64bfb, e020e7ffe8ce423a8bdefec667161217; no voters:
I20240502 07:58:23.194664 6884 raft_consensus.cc:2781] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:23.194731 6884 raft_consensus.cc:482] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:23.194780 6884 raft_consensus.cc:3037] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:23.195756 6884 raft_consensus.cc:504] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } }
I20240502 07:58:23.196172 6884 leader_election.cc:288] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [CANDIDATE]: Term 1 election: Requested vote from peers e020e7ffe8ce423a8bdefec667161217 (127.3.139.69:39399), 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67:37141), 91e82003a656442189a329b29846a841 (127.3.139.65:43429), eda55a8566c447cfa749457825b70688 (127.3.139.66:39925)
I20240502 07:58:23.196429 6154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07d53332bdd34233b3653756770dc9cf" candidate_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e020e7ffe8ce423a8bdefec667161217"
I20240502 07:58:23.196460 5892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07d53332bdd34233b3653756770dc9cf" candidate_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89bf5c0a6b2a4d949851089446f17b96"
I20240502 07:58:23.196449 5628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07d53332bdd34233b3653756770dc9cf" candidate_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e82003a656442189a329b29846a841"
I20240502 07:58:23.196528 6154 raft_consensus.cc:3037] T 07d53332bdd34233b3653756770dc9cf P e020e7ffe8ce423a8bdefec667161217 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:23.196544 5892 raft_consensus.cc:3037] T 07d53332bdd34233b3653756770dc9cf P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:23.196545 5628 raft_consensus.cc:3037] T 07d53332bdd34233b3653756770dc9cf P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:23.197399 5892 raft_consensus.cc:2454] T 07d53332bdd34233b3653756770dc9cf P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d58f6d1ff8cf472f8a9edf3700e64bfb in term 1.
I20240502 07:58:23.197405 5628 raft_consensus.cc:2454] T 07d53332bdd34233b3653756770dc9cf P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d58f6d1ff8cf472f8a9edf3700e64bfb in term 1.
I20240502 07:58:23.197460 6154 raft_consensus.cc:2454] T 07d53332bdd34233b3653756770dc9cf P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d58f6d1ff8cf472f8a9edf3700e64bfb in term 1.
I20240502 07:58:23.197746 6220 leader_election.cc:302] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 89bf5c0a6b2a4d949851089446f17b96, d58f6d1ff8cf472f8a9edf3700e64bfb, e020e7ffe8ce423a8bdefec667161217; no voters:
I20240502 07:58:23.197899 6884 raft_consensus.cc:2781] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:23.197985 6884 raft_consensus.cc:686] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 LEADER]: Becoming Leader. State: Replica: d58f6d1ff8cf472f8a9edf3700e64bfb, State: Running, Role: LEADER
I20240502 07:58:23.198143 5760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07d53332bdd34233b3653756770dc9cf" candidate_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eda55a8566c447cfa749457825b70688" is_pre_election: true
I20240502 07:58:23.198132 6884 consensus_queue.cc:237] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } }
I20240502 07:58:23.198205 5761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07d53332bdd34233b3653756770dc9cf" candidate_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eda55a8566c447cfa749457825b70688"
I20240502 07:58:23.198261 5760 raft_consensus.cc:2454] T 07d53332bdd34233b3653756770dc9cf P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d58f6d1ff8cf472f8a9edf3700e64bfb in term 0.
I20240502 07:58:23.199918 5479 catalog_manager.cc:5484] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb reported cstate change: term changed from 0 to 1, leader changed from <none> to d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70). New cstate: current_term: 1 leader_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:23.255151 6884 raft_consensus.cc:482] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:23.255249 6884 raft_consensus.cc:504] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } }
I20240502 07:58:23.255623 6884 leader_election.cc:288] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68:44287), 91e82003a656442189a329b29846a841 (127.3.139.65:43429), 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67:37141), eda55a8566c447cfa749457825b70688 (127.3.139.66:39925)
I20240502 07:58:23.255913 6023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f3d5d24bc164883adb1a11552500ce6" candidate_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5bc7863cd294f77a932c4a40f98b79d" is_pre_election: true
I20240502 07:58:23.255903 5628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f3d5d24bc164883adb1a11552500ce6" candidate_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e82003a656442189a329b29846a841" is_pre_election: true
I20240502 07:58:23.256039 5628 raft_consensus.cc:2454] T 7f3d5d24bc164883adb1a11552500ce6 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d58f6d1ff8cf472f8a9edf3700e64bfb in term 0.
I20240502 07:58:23.256040 6023 raft_consensus.cc:2454] T 7f3d5d24bc164883adb1a11552500ce6 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d58f6d1ff8cf472f8a9edf3700e64bfb in term 0.
I20240502 07:58:23.256019 5892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f3d5d24bc164883adb1a11552500ce6" candidate_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89bf5c0a6b2a4d949851089446f17b96" is_pre_election: true
I20240502 07:58:23.256038 5761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f3d5d24bc164883adb1a11552500ce6" candidate_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eda55a8566c447cfa749457825b70688" is_pre_election: true
I20240502 07:58:23.256152 5892 raft_consensus.cc:2454] T 7f3d5d24bc164883adb1a11552500ce6 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d58f6d1ff8cf472f8a9edf3700e64bfb in term 0.
I20240502 07:58:23.256152 5761 raft_consensus.cc:2454] T 7f3d5d24bc164883adb1a11552500ce6 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d58f6d1ff8cf472f8a9edf3700e64bfb in term 0.
I20240502 07:58:23.256377 6219 leader_election.cc:302] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 91e82003a656442189a329b29846a841, d58f6d1ff8cf472f8a9edf3700e64bfb, d5bc7863cd294f77a932c4a40f98b79d; no voters:
I20240502 07:58:23.256543 6884 raft_consensus.cc:2781] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:23.256606 6884 raft_consensus.cc:482] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:23.256685 6884 raft_consensus.cc:3037] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:23.257476 6884 raft_consensus.cc:504] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } }
I20240502 07:58:23.257879 6884 leader_election.cc:288] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb [CANDIDATE]: Term 1 election: Requested vote from peers d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68:44287), 91e82003a656442189a329b29846a841 (127.3.139.65:43429), 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67:37141), eda55a8566c447cfa749457825b70688 (127.3.139.66:39925)
I20240502 07:58:23.258147 6023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f3d5d24bc164883adb1a11552500ce6" candidate_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5bc7863cd294f77a932c4a40f98b79d"
I20240502 07:58:23.258229 5628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f3d5d24bc164883adb1a11552500ce6" candidate_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e82003a656442189a329b29846a841"
I20240502 07:58:23.258256 6023 raft_consensus.cc:3037] T 7f3d5d24bc164883adb1a11552500ce6 P d5bc7863cd294f77a932c4a40f98b79d [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:23.258280 5761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f3d5d24bc164883adb1a11552500ce6" candidate_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eda55a8566c447cfa749457825b70688"
I20240502 07:58:23.258285 5892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f3d5d24bc164883adb1a11552500ce6" candidate_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89bf5c0a6b2a4d949851089446f17b96"
I20240502 07:58:23.258319 5628 raft_consensus.cc:3037] T 7f3d5d24bc164883adb1a11552500ce6 P 91e82003a656442189a329b29846a841 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:23.258378 5761 raft_consensus.cc:3037] T 7f3d5d24bc164883adb1a11552500ce6 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:23.258378 5892 raft_consensus.cc:3037] T 7f3d5d24bc164883adb1a11552500ce6 P 89bf5c0a6b2a4d949851089446f17b96 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:23.259163 5628 raft_consensus.cc:2454] T 7f3d5d24bc164883adb1a11552500ce6 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d58f6d1ff8cf472f8a9edf3700e64bfb in term 1.
I20240502 07:58:23.259163 6023 raft_consensus.cc:2454] T 7f3d5d24bc164883adb1a11552500ce6 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d58f6d1ff8cf472f8a9edf3700e64bfb in term 1.
I20240502 07:58:23.259294 5761 raft_consensus.cc:2454] T 7f3d5d24bc164883adb1a11552500ce6 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d58f6d1ff8cf472f8a9edf3700e64bfb in term 1.
I20240502 07:58:23.259294 5892 raft_consensus.cc:2454] T 7f3d5d24bc164883adb1a11552500ce6 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d58f6d1ff8cf472f8a9edf3700e64bfb in term 1.
I20240502 07:58:23.259564 6219 leader_election.cc:302] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 91e82003a656442189a329b29846a841, d58f6d1ff8cf472f8a9edf3700e64bfb, eda55a8566c447cfa749457825b70688; no voters:
I20240502 07:58:23.259723 6884 raft_consensus.cc:2781] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:23.259835 6884 raft_consensus.cc:686] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 LEADER]: Becoming Leader. State: Replica: d58f6d1ff8cf472f8a9edf3700e64bfb, State: Running, Role: LEADER
I20240502 07:58:23.259984 6884 consensus_queue.cc:237] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } }
I20240502 07:58:23.261638 5479 catalog_manager.cc:5484] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb reported cstate change: term changed from 0 to 1, leader changed from <none> to d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70). New cstate: current_term: 1 leader_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:23.310245 6545 raft_consensus.cc:482] T eede21c622d8435ba1efa473ef97b1d1 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:23.310362 6545 raft_consensus.cc:504] T eede21c622d8435ba1efa473ef97b1d1 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } }
I20240502 07:58:23.310956 6545 leader_election.cc:288] T eede21c622d8435ba1efa473ef97b1d1 P eda55a8566c447cfa749457825b70688 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91e82003a656442189a329b29846a841 (127.3.139.65:43429), 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67:37141), d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68:44287), e020e7ffe8ce423a8bdefec667161217 (127.3.139.69:39399)
I20240502 07:58:23.311359 5892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eede21c622d8435ba1efa473ef97b1d1" candidate_uuid: "eda55a8566c447cfa749457825b70688" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89bf5c0a6b2a4d949851089446f17b96" is_pre_election: true
I20240502 07:58:23.311529 5892 raft_consensus.cc:2379] T eede21c622d8435ba1efa473ef97b1d1 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eda55a8566c447cfa749457825b70688 in current term 1: Already voted for candidate 91e82003a656442189a329b29846a841 in this term.
I20240502 07:58:23.311522 5628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eede21c622d8435ba1efa473ef97b1d1" candidate_uuid: "eda55a8566c447cfa749457825b70688" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e82003a656442189a329b29846a841" is_pre_election: true
I20240502 07:58:23.311575 6154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eede21c622d8435ba1efa473ef97b1d1" candidate_uuid: "eda55a8566c447cfa749457825b70688" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e020e7ffe8ce423a8bdefec667161217" is_pre_election: true
I20240502 07:58:23.311728 6154 raft_consensus.cc:2379] T eede21c622d8435ba1efa473ef97b1d1 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eda55a8566c447cfa749457825b70688 in current term 1: Already voted for candidate 91e82003a656442189a329b29846a841 in this term.
I20240502 07:58:23.311959 6023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eede21c622d8435ba1efa473ef97b1d1" candidate_uuid: "eda55a8566c447cfa749457825b70688" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5bc7863cd294f77a932c4a40f98b79d" is_pre_election: true
I20240502 07:58:23.312037 5696 leader_election.cc:302] T eede21c622d8435ba1efa473ef97b1d1 P eda55a8566c447cfa749457825b70688 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 5 voters: 1 yes votes; 3 no votes. yes voters: eda55a8566c447cfa749457825b70688; no voters: 89bf5c0a6b2a4d949851089446f17b96, 91e82003a656442189a329b29846a841, e020e7ffe8ce423a8bdefec667161217
I20240502 07:58:23.312126 6023 raft_consensus.cc:2379] T eede21c622d8435ba1efa473ef97b1d1 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eda55a8566c447cfa749457825b70688 in current term 1: Already voted for candidate 91e82003a656442189a329b29846a841 in this term.
I20240502 07:58:23.312201 6545 raft_consensus.cc:3037] T eede21c622d8435ba1efa473ef97b1d1 P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:23.313069 6545 raft_consensus.cc:2726] T eede21c622d8435ba1efa473ef97b1d1 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20240502 07:58:23.393350 6889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:23.393575 6889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:23.396637 6889 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_4'
I20240502 07:58:23.450932 6740 consensus_queue.cc:1035] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:23.489684 6740 consensus_queue.cc:1035] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:23.492748 6154 raft_consensus.cc:1261] T eede21c622d8435ba1efa473ef97b1d1 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:23.493199 6770 consensus_queue.cc:1035] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:23.493623 6023 raft_consensus.cc:1261] T eede21c622d8435ba1efa473ef97b1d1 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:23.495026 6770 consensus_queue.cc:1035] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:23.497076 5628 raft_consensus.cc:1261] T 07d53332bdd34233b3653756770dc9cf P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:23.497574 6153 raft_consensus.cc:1261] T 07d53332bdd34233b3653756770dc9cf P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:23.497982 6674 consensus_queue.cc:1035] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:23.498515 6674 consensus_queue.cc:1035] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:23.499953 5760 raft_consensus.cc:3037] T 07d53332bdd34233b3653756770dc9cf P eda55a8566c447cfa749457825b70688 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:23.500713 5892 raft_consensus.cc:1261] T 07d53332bdd34233b3653756770dc9cf P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:23.501049 5760 raft_consensus.cc:1261] T 07d53332bdd34233b3653756770dc9cf P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:23.501104 6674 consensus_queue.cc:1035] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:23.503249 6674 consensus_queue.cc:1035] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:23.519156 6923 mvcc.cc:205] Tried to move back new op lower bound from 7023151937537323008 to 7023151936301633536. Current Snapshot: MvccSnapshot[applied={T|T < 7023151937518166016}]
I20240502 07:58:23.520870 5760 raft_consensus.cc:1261] T 7f3d5d24bc164883adb1a11552500ce6 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:23.521961 6023 raft_consensus.cc:1261] T 7f3d5d24bc164883adb1a11552500ce6 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:23.522150 5892 raft_consensus.cc:1261] T 7f3d5d24bc164883adb1a11552500ce6 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:23.522370 6929 consensus_queue.cc:1035] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:23.523887 6674 consensus_queue.cc:1035] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:23.524187 6929 consensus_queue.cc:1035] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:23.525116 6921 mvcc.cc:205] Tried to move back new op lower bound from 7023151937518166016 to 7023151936301633536. Current Snapshot: MvccSnapshot[applied={T|T < 7023151937518166016}]
I20240502 07:58:23.530313 6932 mvcc.cc:205] Tried to move back new op lower bound from 7023151937537323008 to 7023151936301633536. Current Snapshot: MvccSnapshot[applied={T|T < 7023151937518166016}]
I20240502 07:58:23.532088 5628 raft_consensus.cc:1261] T 7f3d5d24bc164883adb1a11552500ce6 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:23.532588 6928 consensus_queue.cc:1035] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:23.541956 6925 mvcc.cc:205] Tried to move back new op lower bound from 7023151937537323008 to 7023151936301633536. Current Snapshot: MvccSnapshot[applied={T|T < 7023151937518166016}]
INSERT report
rows total: 2000
time total: 174.576 ms
time per row: 0.087288 ms
I20240502 07:58:23.609243 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.71:35341
--local_ip_for_outbound_sockets=127.3.139.71
--tserver_master_addrs=127.3.139.126:34225
--webserver_port=45703
--webserver_interface=127.3.139.71
--builtin_ntp_servers=127.3.139.84:41319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240502 07:58:23.711416 6966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:23.711586 6966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:23.711627 6966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:23.714583 6966 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:58:23.714637 6966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:23.714702 6966 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.71
I20240502 07:58:23.718120 6966 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:41319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.71:35341
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.3.139.71
--webserver_port=45703
--tserver_master_addrs=127.3.139.126:34225
--never_fsync=true
--heap_profile_path=/tmp/kudu.6966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.71
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:23.718844 6966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:23.719913 6966 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:23.727375 6974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:23.727628 6971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:23.728078 6966 server_base.cc:1034] running on GCE node
W20240502 07:58:23.728348 6972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:23.728739 6966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:23.729279 6966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:23.730427 6966 hybrid_clock.cc:648] HybridClock initialized: now 1714636703730407 us; error 35 us; skew 500 ppm
I20240502 07:58:23.732756 6966 webserver.cc:460] Webserver started at http://127.3.139.71:45703/ using document root <none> and password file <none>
I20240502 07:58:23.733376 6966 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:23.733469 6966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:23.737017 6966 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20240502 07:58:23.738873 6980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:23.739833 6966 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:23.739931 6966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "27e3fafad7a24bcda362d2197be87017"
format_stamp: "Formatted at 2024-05-02 07:58:17 on dist-test-slave-twcr"
I20240502 07:58:23.740298 6966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:23.763033 6966 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:23.763602 6966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:23.763732 6966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:23.764183 6966 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:23.764987 6966 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:23.765033 6966 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:23.765081 6966 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:23.765106 6966 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:23.773661 6966 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.71:35341
I20240502 07:58:23.773684 7093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.71:35341 every 8 connection(s)
I20240502 07:58:23.774550 6966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240502 07:58:23.782164 7094 heartbeater.cc:347] Connected to a master server at 127.3.139.126:34225
I20240502 07:58:23.782352 7094 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:23.782809 7094 heartbeater.cc:511] Master 127.3.139.126:34225 requested a full tablet report, sending...
I20240502 07:58:23.783836 5483 ts_manager.cc:194] Re-registered known tserver with Master: 27e3fafad7a24bcda362d2197be87017 (127.3.139.71:35341)
I20240502 07:58:23.784211 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 6966
I20240502 07:58:23.784351 3629 test_util.cc:268] Using random seed: -140124769
I20240502 07:58:23.784435 5483 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.71:54669
I20240502 07:58:23.792649 3629 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_0 already exists
I20240502 07:58:23.792819 3629 test_util.cc:268] Using random seed: -140116303
I20240502 07:58:23.801050 3629 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_1 already exists
I20240502 07:58:23.801180 3629 test_util.cc:268] Using random seed: -140107941
I20240502 07:58:23.830566 3629 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_2 already exists
I20240502 07:58:23.830700 3629 test_util.cc:268] Using random seed: -140078422
I20240502 07:58:23.881888 3629 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_3 already exists
I20240502 07:58:23.882063 3629 test_util.cc:268] Using random seed: -140027060
I20240502 07:58:23.930810 3629 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_4 already exists
I20240502 07:58:24.786166 7094 heartbeater.cc:503] Master 127.3.139.126:34225 was elected leader, sending a full tablet report...
I20240502 07:58:24.857262 6002 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:58:24.860023 7028 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:24.872752 6263 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:58:24.887849 6132 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:24.899668 5872 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:24.995810 5740 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:25.009778 5608 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:25.496918 7028 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:25.510262 6002 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:58:25.540069 5608 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:25.542807 5872 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:25.558549 6132 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:25.561010 5740 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:25.736763 6263 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:58:26.158627 6021 consensus_queue.cc:237] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 42, Committed index: 42, Last appended: 1.43, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } }
I20240502 07:58:26.165092 6285 raft_consensus.cc:1261] T 5c8c7437c85943ba9113595c17bf6136 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 43. Preceding OpId from leader: term: 1 index: 44. (index mismatch)
I20240502 07:58:26.165525 7157 consensus_queue.cc:1035] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 44, Last known committed idx: 42, Time since last communication: 0.000s
I20240502 07:58:26.169966 5889 raft_consensus.cc:1261] T 5c8c7437c85943ba9113595c17bf6136 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 43. Preceding OpId from leader: term: 1 index: 44. (index mismatch)
I20240502 07:58:26.172698 7157 consensus_queue.cc:1035] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 44, Last known committed idx: 42, Time since last communication: 0.000s
I20240502 07:58:26.175225 5759 raft_consensus.cc:1261] T 5c8c7437c85943ba9113595c17bf6136 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 43. Preceding OpId from leader: term: 1 index: 44. (index mismatch)
I20240502 07:58:26.176990 7157 consensus_queue.cc:1035] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 44, Last known committed idx: 42, Time since last communication: 0.000s
I20240502 07:58:26.183990 7333 raft_consensus.cc:2932] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d [term 1 LEADER]: Committing config change with OpId 1.44: config changed from index -1 to 44, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:26.186167 5761 raft_consensus.cc:2932] T 5c8c7437c85943ba9113595c17bf6136 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Committing config change with OpId 1.44: config changed from index -1 to 44, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:26.186987 5483 catalog_manager.cc:5484] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d reported cstate change: config changed from index -1 to 44, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New cstate: current_term: 1 leader_uuid: "d5bc7863cd294f77a932c4a40f98b79d" committed_config { opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:26.187150 6284 raft_consensus.cc:2932] T 5c8c7437c85943ba9113595c17bf6136 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Committing config change with OpId 1.44: config changed from index -1 to 44, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:26.204221 5891 raft_consensus.cc:2932] T 5c8c7437c85943ba9113595c17bf6136 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Committing config change with OpId 1.44: config changed from index -1 to 44, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:26.213934 5628 raft_consensus.cc:1261] T 5c8c7437c85943ba9113595c17bf6136 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 43. Preceding OpId from leader: term: 1 index: 44. (index mismatch)
I20240502 07:58:26.227864 6774 consensus_queue.cc:1035] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 44, Last known committed idx: 42, Time since last communication: 0.000s
I20240502 07:58:26.248739 6149 consensus_queue.cc:237] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 65, Committed index: 65, Last appended: 1.65, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } }
I20240502 07:58:26.251341 6283 raft_consensus.cc:1261] T 2a3919f144ed4a27b03e7c4a251499d4 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Refusing update from remote peer e020e7ffe8ce423a8bdefec667161217: Log matching property violated. Preceding OpId in replica: term: 1 index: 65. Preceding OpId from leader: term: 1 index: 66. (index mismatch)
I20240502 07:58:26.252565 6844 consensus_queue.cc:1035] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 66, Last known committed idx: 65, Time since last communication: 0.000s
I20240502 07:58:26.254827 5758 raft_consensus.cc:1261] T 2a3919f144ed4a27b03e7c4a251499d4 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer e020e7ffe8ce423a8bdefec667161217: Log matching property violated. Preceding OpId in replica: term: 1 index: 65. Preceding OpId from leader: term: 1 index: 66. (index mismatch)
I20240502 07:58:26.255200 6022 raft_consensus.cc:1261] T 2a3919f144ed4a27b03e7c4a251499d4 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Refusing update from remote peer e020e7ffe8ce423a8bdefec667161217: Log matching property violated. Preceding OpId in replica: term: 1 index: 65. Preceding OpId from leader: term: 1 index: 66. (index mismatch)
I20240502 07:58:26.256242 7193 consensus_queue.cc:1035] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 66, Last known committed idx: 65, Time since last communication: 0.000s
I20240502 07:58:26.258019 7195 consensus_queue.cc:1035] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 66, Last known committed idx: 65, Time since last communication: 0.000s
W20240502 07:58:26.258045 5958 consensus_peers.cc:485] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d -> Peer 27e3fafad7a24bcda362d2197be87017 (127.3.139.71:35341): Couldn't send request to peer 27e3fafad7a24bcda362d2197be87017. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5c8c7437c85943ba9113595c17bf6136. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:26.259817 6844 raft_consensus.cc:2932] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217 [term 1 LEADER]: Committing config change with OpId 1.66: config changed from index -1 to 66, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:26.262478 5486 catalog_manager.cc:5484] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217 reported cstate change: config changed from index -1 to 66, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New cstate: current_term: 1 leader_uuid: "e020e7ffe8ce423a8bdefec667161217" committed_config { opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:26.266249 5627 raft_consensus.cc:2932] T 5c8c7437c85943ba9113595c17bf6136 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Committing config change with OpId 1.44: config changed from index -1 to 44, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:26.268189 5760 raft_consensus.cc:2932] T 2a3919f144ed4a27b03e7c4a251499d4 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Committing config change with OpId 1.66: config changed from index -1 to 66, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:26.269467 6021 raft_consensus.cc:2932] T 2a3919f144ed4a27b03e7c4a251499d4 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Committing config change with OpId 1.66: config changed from index -1 to 66, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:26.277714 6285 raft_consensus.cc:2932] T 2a3919f144ed4a27b03e7c4a251499d4 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Committing config change with OpId 1.66: config changed from index -1 to 66, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
W20240502 07:58:26.284612 6089 consensus_peers.cc:485] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217 -> Peer 27e3fafad7a24bcda362d2197be87017 (127.3.139.71:35341): Couldn't send request to peer 27e3fafad7a24bcda362d2197be87017. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2a3919f144ed4a27b03e7c4a251499d4. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:26.294895 5627 raft_consensus.cc:1261] T 2a3919f144ed4a27b03e7c4a251499d4 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer e020e7ffe8ce423a8bdefec667161217: Log matching property violated. Preceding OpId in replica: term: 1 index: 65. Preceding OpId from leader: term: 1 index: 66. (index mismatch)
I20240502 07:58:26.301566 7195 consensus_queue.cc:1035] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 66, Last known committed idx: 65, Time since last communication: 0.000s
I20240502 07:58:26.352226 5628 raft_consensus.cc:2932] T 2a3919f144ed4a27b03e7c4a251499d4 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Committing config change with OpId 1.66: config changed from index -1 to 66, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:26.415697 6022 consensus_queue.cc:237] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 45, Committed index: 45, Last appended: 1.45, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 46 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } }
I20240502 07:58:26.417564 6285 raft_consensus.cc:1261] T 5b87554a301044caac956bc5c7f137e8 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 45. Preceding OpId from leader: term: 1 index: 46. (index mismatch)
I20240502 07:58:26.417938 7332 consensus_queue.cc:1035] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 46, Last known committed idx: 45, Time since last communication: 0.000s
W20240502 07:58:26.418072 5958 consensus_peers.cc:485] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d -> Peer 27e3fafad7a24bcda362d2197be87017 (127.3.139.71:35341): Couldn't send request to peer 27e3fafad7a24bcda362d2197be87017. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5b87554a301044caac956bc5c7f137e8. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:26.421065 5892 raft_consensus.cc:1261] T 5b87554a301044caac956bc5c7f137e8 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 45. Preceding OpId from leader: term: 1 index: 46. (index mismatch)
I20240502 07:58:26.423535 7332 consensus_queue.cc:1035] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 46, Last known committed idx: 45, Time since last communication: 0.000s
I20240502 07:58:26.431351 7332 raft_consensus.cc:2932] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d [term 1 LEADER]: Committing config change with OpId 1.46: config changed from index -1 to 46, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 46 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:26.434305 6284 raft_consensus.cc:2932] T 5b87554a301044caac956bc5c7f137e8 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Committing config change with OpId 1.46: config changed from index -1 to 46, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 46 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:26.436280 5891 raft_consensus.cc:2932] T 5b87554a301044caac956bc5c7f137e8 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Committing config change with OpId 1.46: config changed from index -1 to 46, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 46 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:26.451400 5628 raft_consensus.cc:1261] T 5b87554a301044caac956bc5c7f137e8 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 45. Preceding OpId from leader: term: 1 index: 46. (index mismatch)
I20240502 07:58:26.458950 6774 consensus_queue.cc:1035] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 46, Last known committed idx: 45, Time since last communication: 0.000s
I20240502 07:58:26.460537 5630 raft_consensus.cc:2932] T 5b87554a301044caac956bc5c7f137e8 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Committing config change with OpId 1.46: config changed from index -1 to 46, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 46 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:26.481161 5759 raft_consensus.cc:1261] T 5b87554a301044caac956bc5c7f137e8 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 45. Preceding OpId from leader: term: 1 index: 46. (index mismatch)
I20240502 07:58:26.496723 7274 consensus_queue.cc:1035] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 46, Last known committed idx: 45, Time since last communication: 0.000s
I20240502 07:58:26.512291 5482 catalog_manager.cc:5484] T 5b87554a301044caac956bc5c7f137e8 P 89bf5c0a6b2a4d949851089446f17b96 reported cstate change: config changed from index -1 to 46, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New cstate: current_term: 1 leader_uuid: "d5bc7863cd294f77a932c4a40f98b79d" committed_config { opid_index: 46 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:26.512439 5759 raft_consensus.cc:2932] T 5b87554a301044caac956bc5c7f137e8 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Committing config change with OpId 1.46: config changed from index -1 to 46, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 46 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:26.591605 6283 consensus_queue.cc:237] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 44, Committed index: 44, Last appended: 1.44, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } }
I20240502 07:58:26.594262 6149 raft_consensus.cc:1261] T d41a2cb7b69e40d9a681384e5ad0a451 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 44. Preceding OpId from leader: term: 1 index: 45. (index mismatch)
I20240502 07:58:26.594681 7212 consensus_queue.cc:1035] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 45, Last known committed idx: 44, Time since last communication: 0.000s
I20240502 07:58:26.601259 6022 raft_consensus.cc:1261] T d41a2cb7b69e40d9a681384e5ad0a451 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 44. Preceding OpId from leader: term: 1 index: 45. (index mismatch)
I20240502 07:58:26.605870 5627 raft_consensus.cc:1261] T d41a2cb7b69e40d9a681384e5ad0a451 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 44. Preceding OpId from leader: term: 1 index: 45. (index mismatch)
I20240502 07:58:26.608866 7212 consensus_queue.cc:1035] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 45, Last known committed idx: 44, Time since last communication: 0.000s
W20240502 07:58:26.636061 6220 consensus_peers.cc:485] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb -> Peer 27e3fafad7a24bcda362d2197be87017 (127.3.139.71:35341): Couldn't send request to peer 27e3fafad7a24bcda362d2197be87017. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d41a2cb7b69e40d9a681384e5ad0a451. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:26.654513 7379 ts_tablet_manager.cc:902] T 5c8c7437c85943ba9113595c17bf6136 P 27e3fafad7a24bcda362d2197be87017: Initiating tablet copy from peer d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68:44287)
I20240502 07:58:26.655344 7379 tablet_copy_client.cc:308] T 5c8c7437c85943ba9113595c17bf6136 P 27e3fafad7a24bcda362d2197be87017: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.68:44287
I20240502 07:58:26.664145 6043 tablet_copy_service.cc:140] P d5bc7863cd294f77a932c4a40f98b79d: Received BeginTabletCopySession request for tablet 5c8c7437c85943ba9113595c17bf6136 from peer 27e3fafad7a24bcda362d2197be87017 ({username='slave'} at 127.3.139.71:34815)
I20240502 07:58:26.664283 6043 tablet_copy_service.cc:161] P d5bc7863cd294f77a932c4a40f98b79d: Beginning new tablet copy session on tablet 5c8c7437c85943ba9113595c17bf6136 from peer 27e3fafad7a24bcda362d2197be87017 at {username='slave'} at 127.3.139.71:34815: session id = 27e3fafad7a24bcda362d2197be87017-5c8c7437c85943ba9113595c17bf6136
I20240502 07:58:26.665680 6043 tablet_copy_source_session.cc:205] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:26.669359 7379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c8c7437c85943ba9113595c17bf6136. 1 dirs total, 0 dirs full, 0 dirs failed
W20240502 07:58:26.669627 6219 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.139.68:44287, user_credentials={real_user=slave}} blocked reactor thread for 60794.6us
I20240502 07:58:26.671587 5891 raft_consensus.cc:1261] T d41a2cb7b69e40d9a681384e5ad0a451 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 44. Preceding OpId from leader: term: 1 index: 45. (index mismatch)
I20240502 07:58:26.683643 7379 tablet_copy_client.cc:783] T 5c8c7437c85943ba9113595c17bf6136 P 27e3fafad7a24bcda362d2197be87017: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:26.683826 7379 tablet_copy_client.cc:647] T 5c8c7437c85943ba9113595c17bf6136 P 27e3fafad7a24bcda362d2197be87017: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:26.690527 7379 tablet_copy_client.cc:523] T 5c8c7437c85943ba9113595c17bf6136 P 27e3fafad7a24bcda362d2197be87017: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:26.692636 7379 tablet_bootstrap.cc:492] T 5c8c7437c85943ba9113595c17bf6136 P 27e3fafad7a24bcda362d2197be87017: Bootstrap starting.
I20240502 07:58:26.692579 7212 consensus_queue.cc:1035] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 45, Last known committed idx: 44, Time since last communication: 0.000s
I20240502 07:58:26.712476 7379 log.cc:824] T 5c8c7437c85943ba9113595c17bf6136 P 27e3fafad7a24bcda362d2197be87017: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:26.714800 6928 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 LEADER]: Committing config change with OpId 1.45: config changed from index -1 to 45, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:26.716697 6153 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.45: config changed from index -1 to 45, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:26.719447 5483 catalog_manager.cc:5484] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb reported cstate change: config changed from index -1 to 45, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New cstate: current_term: 1 leader_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" committed_config { opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:26.731870 7308 consensus_queue.cc:1035] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 45, Last known committed idx: 44, Time since last communication: 0.000s
I20240502 07:58:26.737215 6022 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Committing config change with OpId 1.45: config changed from index -1 to 45, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:26.739555 5890 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Committing config change with OpId 1.45: config changed from index -1 to 45, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:26.755743 5630 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Committing config change with OpId 1.45: config changed from index -1 to 45, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:26.772420 7386 ts_tablet_manager.cc:902] T 2a3919f144ed4a27b03e7c4a251499d4 P 27e3fafad7a24bcda362d2197be87017: Initiating tablet copy from peer e020e7ffe8ce423a8bdefec667161217 (127.3.139.69:39399)
I20240502 07:58:26.772925 7386 tablet_copy_client.cc:308] T 2a3919f144ed4a27b03e7c4a251499d4 P 27e3fafad7a24bcda362d2197be87017: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.69:39399
I20240502 07:58:26.794772 6174 tablet_copy_service.cc:140] P e020e7ffe8ce423a8bdefec667161217: Received BeginTabletCopySession request for tablet 2a3919f144ed4a27b03e7c4a251499d4 from peer 27e3fafad7a24bcda362d2197be87017 ({username='slave'} at 127.3.139.71:53701)
I20240502 07:58:26.794912 6174 tablet_copy_service.cc:161] P e020e7ffe8ce423a8bdefec667161217: Beginning new tablet copy session on tablet 2a3919f144ed4a27b03e7c4a251499d4 from peer 27e3fafad7a24bcda362d2197be87017 at {username='slave'} at 127.3.139.71:53701: session id = 27e3fafad7a24bcda362d2197be87017-2a3919f144ed4a27b03e7c4a251499d4
I20240502 07:58:26.796190 6174 tablet_copy_source_session.cc:205] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:26.805382 7386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a3919f144ed4a27b03e7c4a251499d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:26.816429 7399 ts_tablet_manager.cc:902] T 5b87554a301044caac956bc5c7f137e8 P 27e3fafad7a24bcda362d2197be87017: Initiating tablet copy from peer d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68:44287)
I20240502 07:58:26.861133 7379 tablet_bootstrap.cc:492] T 5c8c7437c85943ba9113595c17bf6136 P 27e3fafad7a24bcda362d2197be87017: Bootstrap replayed 1/1 log segments. Stats: ops{read=57 overwritten=0 applied=56 ignored=0} inserts{seen=687 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240502 07:58:26.861703 7379 tablet_bootstrap.cc:492] T 5c8c7437c85943ba9113595c17bf6136 P 27e3fafad7a24bcda362d2197be87017: Bootstrap complete.
I20240502 07:58:26.862380 7379 ts_tablet_manager.cc:1367] T 5c8c7437c85943ba9113595c17bf6136 P 27e3fafad7a24bcda362d2197be87017: Time spent bootstrapping tablet: real 0.170s user 0.063s sys 0.009s
I20240502 07:58:26.865031 7379 raft_consensus.cc:348] T 5c8c7437c85943ba9113595c17bf6136 P 27e3fafad7a24bcda362d2197be87017 [term 1 LEARNER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } }
I20240502 07:58:26.866660 7379 raft_consensus.cc:729] T 5c8c7437c85943ba9113595c17bf6136 P 27e3fafad7a24bcda362d2197be87017 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 27e3fafad7a24bcda362d2197be87017, State: Initialized, Role: LEARNER
I20240502 07:58:26.867090 7379 consensus_queue.cc:260] T 5c8c7437c85943ba9113595c17bf6136 P 27e3fafad7a24bcda362d2197be87017 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 56, Last appended: 1.57, Last appended by leader: 57, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } }
I20240502 07:58:26.868221 7379 ts_tablet_manager.cc:1398] T 5c8c7437c85943ba9113595c17bf6136 P 27e3fafad7a24bcda362d2197be87017: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20240502 07:58:26.871553 6043 tablet_copy_service.cc:342] P d5bc7863cd294f77a932c4a40f98b79d: Request end of tablet copy session 27e3fafad7a24bcda362d2197be87017-5c8c7437c85943ba9113595c17bf6136 received from {username='slave'} at 127.3.139.71:34815
I20240502 07:58:26.871651 6043 tablet_copy_service.cc:433] P d5bc7863cd294f77a932c4a40f98b79d: ending tablet copy session 27e3fafad7a24bcda362d2197be87017-5c8c7437c85943ba9113595c17bf6136 on tablet 5c8c7437c85943ba9113595c17bf6136 with peer 27e3fafad7a24bcda362d2197be87017
I20240502 07:58:26.878794 7399 tablet_copy_client.cc:308] T 5b87554a301044caac956bc5c7f137e8 P 27e3fafad7a24bcda362d2197be87017: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.68:44287
I20240502 07:58:26.879179 6043 tablet_copy_service.cc:140] P d5bc7863cd294f77a932c4a40f98b79d: Received BeginTabletCopySession request for tablet 5b87554a301044caac956bc5c7f137e8 from peer 27e3fafad7a24bcda362d2197be87017 ({username='slave'} at 127.3.139.71:34815)
I20240502 07:58:26.879268 6043 tablet_copy_service.cc:161] P d5bc7863cd294f77a932c4a40f98b79d: Beginning new tablet copy session on tablet 5b87554a301044caac956bc5c7f137e8 from peer 27e3fafad7a24bcda362d2197be87017 at {username='slave'} at 127.3.139.71:34815: session id = 27e3fafad7a24bcda362d2197be87017-5b87554a301044caac956bc5c7f137e8
I20240502 07:58:26.880125 6043 tablet_copy_source_session.cc:205] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:26.895009 7386 tablet_copy_client.cc:783] T 2a3919f144ed4a27b03e7c4a251499d4 P 27e3fafad7a24bcda362d2197be87017: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:26.895490 7386 tablet_copy_client.cc:647] T 2a3919f144ed4a27b03e7c4a251499d4 P 27e3fafad7a24bcda362d2197be87017: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:26.897122 7386 tablet_copy_client.cc:523] T 2a3919f144ed4a27b03e7c4a251499d4 P 27e3fafad7a24bcda362d2197be87017: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:26.898363 7386 tablet_bootstrap.cc:492] T 2a3919f144ed4a27b03e7c4a251499d4 P 27e3fafad7a24bcda362d2197be87017: Bootstrap starting.
I20240502 07:58:26.898725 7399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b87554a301044caac956bc5c7f137e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:26.952627 7399 tablet_copy_client.cc:783] T 5b87554a301044caac956bc5c7f137e8 P 27e3fafad7a24bcda362d2197be87017: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:26.952843 7399 tablet_copy_client.cc:647] T 5b87554a301044caac956bc5c7f137e8 P 27e3fafad7a24bcda362d2197be87017: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:26.962839 7399 tablet_copy_client.cc:523] T 5b87554a301044caac956bc5c7f137e8 P 27e3fafad7a24bcda362d2197be87017: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:26.964794 7399 tablet_bootstrap.cc:492] T 5b87554a301044caac956bc5c7f137e8 P 27e3fafad7a24bcda362d2197be87017: Bootstrap starting.
I20240502 07:58:26.968647 5630 consensus_queue.cc:237] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 81, Committed index: 81, Last appended: 1.82, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 83 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } }
I20240502 07:58:26.970296 6153 raft_consensus.cc:1261] T 96dcb06388374514951f24393f288eca P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 82. Preceding OpId from leader: term: 1 index: 83. (index mismatch)
I20240502 07:58:26.970731 6770 consensus_queue.cc:1035] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 83, Last known committed idx: 81, Time since last communication: 0.000s
I20240502 07:58:26.971235 6285 raft_consensus.cc:1261] T 96dcb06388374514951f24393f288eca P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 82. Preceding OpId from leader: term: 1 index: 83. (index mismatch)
I20240502 07:58:26.971596 6770 consensus_queue.cc:1035] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 83, Last known committed idx: 81, Time since last communication: 0.000s
I20240502 07:58:26.972981 5759 raft_consensus.cc:1261] T 96dcb06388374514951f24393f288eca P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 81. Preceding OpId from leader: term: 1 index: 83. (index mismatch)
I20240502 07:58:26.973840 6770 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [term 1 LEADER]: Committing config change with OpId 1.83: config changed from index -1 to 83, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 83 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:26.974493 6283 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Committing config change with OpId 1.83: config changed from index -1 to 83, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 83 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:26.975884 6020 raft_consensus.cc:1261] T 96dcb06388374514951f24393f288eca P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 82. Preceding OpId from leader: term: 1 index: 83. (index mismatch)
I20240502 07:58:26.976864 6148 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.83: config changed from index -1 to 83, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 83 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:26.977133 5480 catalog_manager.cc:5484] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 reported cstate change: config changed from index -1 to 83, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New cstate: current_term: 1 leader_uuid: "91e82003a656442189a329b29846a841" committed_config { opid_index: 83 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:26.980798 6770 consensus_queue.cc:1035] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 83, Last known committed idx: 81, Time since last communication: 0.000s
I20240502 07:58:26.989373 5759 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Committing config change with OpId 1.83: config changed from index -1 to 83, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 83 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:26.994649 5758 raft_consensus.cc:1203] T 96dcb06388374514951f24393f288eca P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.81->[1.82-1.82] Dedup: 1.82->[]
I20240502 07:58:27.008782 7197 consensus_queue.cc:1035] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 83, Last known committed idx: 81, Time since last communication: 0.000s
I20240502 07:58:27.019137 6021 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Committing config change with OpId 1.83: config changed from index -1 to 83, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 83 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
W20240502 07:58:27.092239 5565 consensus_peers.cc:485] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 -> Peer 27e3fafad7a24bcda362d2197be87017 (127.3.139.71:35341): Couldn't send request to peer 27e3fafad7a24bcda362d2197be87017. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 96dcb06388374514951f24393f288eca. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:27.101624 7379 ts_tablet_manager.cc:902] T d41a2cb7b69e40d9a681384e5ad0a451 P 27e3fafad7a24bcda362d2197be87017: Initiating tablet copy from peer d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70:41047)
I20240502 07:58:27.102106 7379 tablet_copy_client.cc:308] T d41a2cb7b69e40d9a681384e5ad0a451 P 27e3fafad7a24bcda362d2197be87017: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.70:41047
I20240502 07:58:27.166831 6303 tablet_copy_service.cc:140] P d58f6d1ff8cf472f8a9edf3700e64bfb: Received BeginTabletCopySession request for tablet d41a2cb7b69e40d9a681384e5ad0a451 from peer 27e3fafad7a24bcda362d2197be87017 ({username='slave'} at 127.3.139.71:33001)
I20240502 07:58:27.166988 6303 tablet_copy_service.cc:161] P d58f6d1ff8cf472f8a9edf3700e64bfb: Beginning new tablet copy session on tablet d41a2cb7b69e40d9a681384e5ad0a451 from peer 27e3fafad7a24bcda362d2197be87017 at {username='slave'} at 127.3.139.71:33001: session id = 27e3fafad7a24bcda362d2197be87017-d41a2cb7b69e40d9a681384e5ad0a451
I20240502 07:58:27.168406 6303 tablet_copy_source_session.cc:205] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:27.169533 7379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d41a2cb7b69e40d9a681384e5ad0a451. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:27.172277 7379 tablet_copy_client.cc:783] T d41a2cb7b69e40d9a681384e5ad0a451 P 27e3fafad7a24bcda362d2197be87017: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:27.172423 7379 tablet_copy_client.cc:647] T d41a2cb7b69e40d9a681384e5ad0a451 P 27e3fafad7a24bcda362d2197be87017: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:27.175679 7379 tablet_copy_client.cc:523] T d41a2cb7b69e40d9a681384e5ad0a451 P 27e3fafad7a24bcda362d2197be87017: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:27.176877 7379 tablet_bootstrap.cc:492] T d41a2cb7b69e40d9a681384e5ad0a451 P 27e3fafad7a24bcda362d2197be87017: Bootstrap starting.
I20240502 07:58:27.239553 7046 raft_consensus.cc:1203] T 5c8c7437c85943ba9113595c17bf6136 P 27e3fafad7a24bcda362d2197be87017 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.43->[1.44-1.65] Dedup: 1.57->[1.58-1.65]
I20240502 07:58:27.427742 7332 raft_consensus.cc:1050] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d: attempting to promote NON_VOTER 27e3fafad7a24bcda362d2197be87017 to VOTER
I20240502 07:58:27.428241 7332 consensus_queue.cc:237] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 65, Committed index: 65, Last appended: 1.66, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 67 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } }
I20240502 07:58:27.429636 7046 raft_consensus.cc:1261] T 5c8c7437c85943ba9113595c17bf6136 P 27e3fafad7a24bcda362d2197be87017 [term 1 LEARNER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 66. Preceding OpId from leader: term: 1 index: 67. (index mismatch)
I20240502 07:58:27.430413 7157 consensus_queue.cc:1035] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 67, Last known committed idx: 65, Time since last communication: 0.000s
I20240502 07:58:27.430687 6283 raft_consensus.cc:1261] T 5c8c7437c85943ba9113595c17bf6136 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 66. Preceding OpId from leader: term: 1 index: 67. (index mismatch)
I20240502 07:58:27.430689 5890 raft_consensus.cc:1261] T 5c8c7437c85943ba9113595c17bf6136 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 66. Preceding OpId from leader: term: 1 index: 67. (index mismatch)
I20240502 07:58:27.431880 7368 consensus_queue.cc:1035] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 67, Last known committed idx: 65, Time since last communication: 0.000s
I20240502 07:58:27.432746 7333 consensus_queue.cc:1035] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 67, Last known committed idx: 65, Time since last communication: 0.000s
I20240502 07:58:27.434983 7274 raft_consensus.cc:2932] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d [term 1 LEADER]: Committing config change with OpId 1.67: config changed from index 44 to 67, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 67 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:27.437192 7046 raft_consensus.cc:2932] T 5c8c7437c85943ba9113595c17bf6136 P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Committing config change with OpId 1.67: config changed from index 44 to 67, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 67 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:27.438640 5483 catalog_manager.cc:5484] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d reported cstate change: config changed from index 44 to 67, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d5bc7863cd294f77a932c4a40f98b79d" committed_config { opid_index: 67 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:58:27.439874 6283 raft_consensus.cc:2932] T 5c8c7437c85943ba9113595c17bf6136 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Committing config change with OpId 1.67: config changed from index 44 to 67, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 67 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:27.445997 5890 raft_consensus.cc:2932] T 5c8c7437c85943ba9113595c17bf6136 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Committing config change with OpId 1.67: config changed from index 44 to 67, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 67 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:27.448848 5760 raft_consensus.cc:1261] T 5c8c7437c85943ba9113595c17bf6136 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 66. Preceding OpId from leader: term: 1 index: 67. (index mismatch)
I20240502 07:58:27.458447 7368 consensus_queue.cc:1035] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 67, Last known committed idx: 65, Time since last communication: 0.000s
I20240502 07:58:27.465709 5757 raft_consensus.cc:2932] T 5c8c7437c85943ba9113595c17bf6136 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Committing config change with OpId 1.67: config changed from index 44 to 67, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 67 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:27.479511 5630 raft_consensus.cc:1261] T 5c8c7437c85943ba9113595c17bf6136 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 66. Preceding OpId from leader: term: 1 index: 67. (index mismatch)
I20240502 07:58:27.515208 7332 consensus_queue.cc:1035] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 67, Last known committed idx: 65, Time since last communication: 0.000s
I20240502 07:58:27.589154 7399 tablet_bootstrap.cc:492] T 5b87554a301044caac956bc5c7f137e8 P 27e3fafad7a24bcda362d2197be87017: Bootstrap replayed 1/1 log segments. Stats: ops{read=52 overwritten=0 applied=52 ignored=0} inserts{seen=683 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:58:27.589457 7399 tablet_bootstrap.cc:492] T 5b87554a301044caac956bc5c7f137e8 P 27e3fafad7a24bcda362d2197be87017: Bootstrap complete.
I20240502 07:58:27.589643 7399 ts_tablet_manager.cc:1367] T 5b87554a301044caac956bc5c7f137e8 P 27e3fafad7a24bcda362d2197be87017: Time spent bootstrapping tablet: real 0.625s user 0.053s sys 0.003s
I20240502 07:58:27.590214 7399 raft_consensus.cc:348] T 5b87554a301044caac956bc5c7f137e8 P 27e3fafad7a24bcda362d2197be87017 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 46 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } }
I20240502 07:58:27.593739 7399 raft_consensus.cc:729] T 5b87554a301044caac956bc5c7f137e8 P 27e3fafad7a24bcda362d2197be87017 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 27e3fafad7a24bcda362d2197be87017, State: Initialized, Role: LEARNER
I20240502 07:58:27.595427 7399 consensus_queue.cc:260] T 5b87554a301044caac956bc5c7f137e8 P 27e3fafad7a24bcda362d2197be87017 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 52, Last appended: 1.52, Last appended by leader: 52, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 46 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } }
I20240502 07:58:27.597947 7399 ts_tablet_manager.cc:1398] T 5b87554a301044caac956bc5c7f137e8 P 27e3fafad7a24bcda362d2197be87017: Time spent starting tablet: real 0.008s user 0.003s sys 0.000s
I20240502 07:58:27.601256 6043 tablet_copy_service.cc:342] P d5bc7863cd294f77a932c4a40f98b79d: Request end of tablet copy session 27e3fafad7a24bcda362d2197be87017-5b87554a301044caac956bc5c7f137e8 received from {username='slave'} at 127.3.139.71:34815
I20240502 07:58:27.601332 6043 tablet_copy_service.cc:433] P d5bc7863cd294f77a932c4a40f98b79d: ending tablet copy session 27e3fafad7a24bcda362d2197be87017-5b87554a301044caac956bc5c7f137e8 on tablet 5b87554a301044caac956bc5c7f137e8 with peer 27e3fafad7a24bcda362d2197be87017
I20240502 07:58:27.603253 7399 ts_tablet_manager.cc:902] T 96dcb06388374514951f24393f288eca P 27e3fafad7a24bcda362d2197be87017: Initiating tablet copy from peer 91e82003a656442189a329b29846a841 (127.3.139.65:43429)
I20240502 07:58:27.603641 7399 tablet_copy_client.cc:308] T 96dcb06388374514951f24393f288eca P 27e3fafad7a24bcda362d2197be87017: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.65:43429
I20240502 07:58:27.616829 5629 raft_consensus.cc:2932] T 5c8c7437c85943ba9113595c17bf6136 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Committing config change with OpId 1.67: config changed from index 44 to 67, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 67 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:27.621774 5650 tablet_copy_service.cc:140] P 91e82003a656442189a329b29846a841: Received BeginTabletCopySession request for tablet 96dcb06388374514951f24393f288eca from peer 27e3fafad7a24bcda362d2197be87017 ({username='slave'} at 127.3.139.71:34345)
I20240502 07:58:27.621907 5650 tablet_copy_service.cc:161] P 91e82003a656442189a329b29846a841: Beginning new tablet copy session on tablet 96dcb06388374514951f24393f288eca from peer 27e3fafad7a24bcda362d2197be87017 at {username='slave'} at 127.3.139.71:34345: session id = 27e3fafad7a24bcda362d2197be87017-96dcb06388374514951f24393f288eca
I20240502 07:58:27.623183 5650 tablet_copy_source_session.cc:205] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:27.624611 7399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96dcb06388374514951f24393f288eca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:27.626539 7399 tablet_copy_client.cc:783] T 96dcb06388374514951f24393f288eca P 27e3fafad7a24bcda362d2197be87017: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:27.626673 7399 tablet_copy_client.cc:647] T 96dcb06388374514951f24393f288eca P 27e3fafad7a24bcda362d2197be87017: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:27.627979 7386 tablet_bootstrap.cc:492] T 2a3919f144ed4a27b03e7c4a251499d4 P 27e3fafad7a24bcda362d2197be87017: Bootstrap replayed 1/1 log segments. Stats: ops{read=93 overwritten=0 applied=93 ignored=0} inserts{seen=765 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:58:27.628360 7386 tablet_bootstrap.cc:492] T 2a3919f144ed4a27b03e7c4a251499d4 P 27e3fafad7a24bcda362d2197be87017: Bootstrap complete.
I20240502 07:58:27.628578 7386 ts_tablet_manager.cc:1367] T 2a3919f144ed4a27b03e7c4a251499d4 P 27e3fafad7a24bcda362d2197be87017: Time spent bootstrapping tablet: real 0.730s user 0.075s sys 0.014s
I20240502 07:58:27.628983 7386 raft_consensus.cc:348] T 2a3919f144ed4a27b03e7c4a251499d4 P 27e3fafad7a24bcda362d2197be87017 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } }
I20240502 07:58:27.629091 7386 raft_consensus.cc:729] T 2a3919f144ed4a27b03e7c4a251499d4 P 27e3fafad7a24bcda362d2197be87017 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 27e3fafad7a24bcda362d2197be87017, State: Initialized, Role: LEARNER
I20240502 07:58:27.629405 7386 consensus_queue.cc:260] T 2a3919f144ed4a27b03e7c4a251499d4 P 27e3fafad7a24bcda362d2197be87017 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 93, Last appended: 1.93, Last appended by leader: 93, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } }
I20240502 07:58:27.632289 7386 ts_tablet_manager.cc:1398] T 2a3919f144ed4a27b03e7c4a251499d4 P 27e3fafad7a24bcda362d2197be87017: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:58:27.632354 7399 tablet_copy_client.cc:523] T 96dcb06388374514951f24393f288eca P 27e3fafad7a24bcda362d2197be87017: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:27.633404 7399 tablet_bootstrap.cc:492] T 96dcb06388374514951f24393f288eca P 27e3fafad7a24bcda362d2197be87017: Bootstrap starting.
I20240502 07:58:27.635623 6174 tablet_copy_service.cc:342] P e020e7ffe8ce423a8bdefec667161217: Request end of tablet copy session 27e3fafad7a24bcda362d2197be87017-2a3919f144ed4a27b03e7c4a251499d4 received from {username='slave'} at 127.3.139.71:53701
I20240502 07:58:27.635715 6174 tablet_copy_service.cc:433] P e020e7ffe8ce423a8bdefec667161217: ending tablet copy session 27e3fafad7a24bcda362d2197be87017-2a3919f144ed4a27b03e7c4a251499d4 on tablet 2a3919f144ed4a27b03e7c4a251499d4 with peer 27e3fafad7a24bcda362d2197be87017
I20240502 07:58:27.666157 6022 consensus_queue.cc:237] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 67, Committed index: 67, Last appended: 1.68, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 69 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } }
I20240502 07:58:27.667311 7046 raft_consensus.cc:1261] T 5c8c7437c85943ba9113595c17bf6136 P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 68. Preceding OpId from leader: term: 1 index: 69. (index mismatch)
I20240502 07:58:27.668618 7274 consensus_queue.cc:1035] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 69, Last known committed idx: 67, Time since last communication: 0.000s
I20240502 07:58:27.675398 7046 raft_consensus.cc:1203] T 5b87554a301044caac956bc5c7f137e8 P 27e3fafad7a24bcda362d2197be87017 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.45->[1.46-1.62] Dedup: 1.52->[1.53-1.62]
I20240502 07:58:27.717423 6282 raft_consensus.cc:1261] T 5c8c7437c85943ba9113595c17bf6136 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 67. Preceding OpId from leader: term: 1 index: 69. (index mismatch)
I20240502 07:58:27.717844 6774 consensus_queue.cc:1035] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 69, Last known committed idx: 67, Time since last communication: 0.000s
I20240502 07:58:27.720893 6774 raft_consensus.cc:2932] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d [term 1 LEADER]: Committing config change with OpId 1.69: config changed from index 67 to 69, VOTER 91e82003a656442189a329b29846a841 (127.3.139.65) evicted. New config: { opid_index: 69 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:27.724264 7048 raft_consensus.cc:2932] T 5c8c7437c85943ba9113595c17bf6136 P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Committing config change with OpId 1.69: config changed from index 67 to 69, VOTER 91e82003a656442189a329b29846a841 (127.3.139.65) evicted. New config: { opid_index: 69 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:27.727525 5480 catalog_manager.cc:5484] T 5c8c7437c85943ba9113595c17bf6136 P 27e3fafad7a24bcda362d2197be87017 reported cstate change: config changed from index 67 to 69, VOTER 91e82003a656442189a329b29846a841 (127.3.139.65) evicted. New cstate: current_term: 1 leader_uuid: "d5bc7863cd294f77a932c4a40f98b79d" committed_config { opid_index: 69 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:27.728185 5473 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 5c8c7437c85943ba9113595c17bf6136 with cas_config_opid_index 67: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:58:27.728543 6285 raft_consensus.cc:2932] T 5c8c7437c85943ba9113595c17bf6136 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Committing config change with OpId 1.69: config changed from index 67 to 69, VOTER 91e82003a656442189a329b29846a841 (127.3.139.65) evicted. New config: { opid_index: 69 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:27.733135 6283 raft_consensus.cc:1203] T 5c8c7437c85943ba9113595c17bf6136 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.67->[1.68-1.68] Dedup: 1.68->[]
I20240502 07:58:27.734108 5608 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c8c7437c85943ba9113595c17bf6136 with delete_type TABLET_DATA_TOMBSTONED (TS 91e82003a656442189a329b29846a841 not found in new config with opid_index 69) from {username='slave'} at 127.0.0.1:54448
I20240502 07:58:27.760910 7048 raft_consensus.cc:1203] T 2a3919f144ed4a27b03e7c4a251499d4 P 27e3fafad7a24bcda362d2197be87017 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.65->[1.66-1.118] Dedup: 1.93->[1.94-1.118]
I20240502 07:58:27.784475 5760 raft_consensus.cc:1261] T 5c8c7437c85943ba9113595c17bf6136 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 67. Preceding OpId from leader: term: 1 index: 69. (index mismatch)
I20240502 07:58:27.786123 5890 raft_consensus.cc:1261] T 5c8c7437c85943ba9113595c17bf6136 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 68. Preceding OpId from leader: term: 1 index: 69. (index mismatch)
I20240502 07:58:27.789093 7333 consensus_queue.cc:1035] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 69, Last known committed idx: 67, Time since last communication: 0.000s
I20240502 07:58:27.790102 7399 tablet_bootstrap.cc:492] T 96dcb06388374514951f24393f288eca P 27e3fafad7a24bcda362d2197be87017: Bootstrap replayed 1/1 log segments. Stats: ops{read=112 overwritten=0 applied=112 ignored=0} inserts{seen=797 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:58:27.790771 7399 tablet_bootstrap.cc:492] T 96dcb06388374514951f24393f288eca P 27e3fafad7a24bcda362d2197be87017: Bootstrap complete.
I20240502 07:58:27.791270 7399 ts_tablet_manager.cc:1367] T 96dcb06388374514951f24393f288eca P 27e3fafad7a24bcda362d2197be87017: Time spent bootstrapping tablet: real 0.158s user 0.100s sys 0.000s
I20240502 07:58:27.791692 7399 raft_consensus.cc:348] T 96dcb06388374514951f24393f288eca P 27e3fafad7a24bcda362d2197be87017 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 83 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } }
I20240502 07:58:27.791810 7399 raft_consensus.cc:729] T 96dcb06388374514951f24393f288eca P 27e3fafad7a24bcda362d2197be87017 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 27e3fafad7a24bcda362d2197be87017, State: Initialized, Role: LEARNER
I20240502 07:58:27.791924 7399 consensus_queue.cc:260] T 96dcb06388374514951f24393f288eca P 27e3fafad7a24bcda362d2197be87017 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 112, Last appended: 1.112, Last appended by leader: 112, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 83 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } }
I20240502 07:58:27.792187 7399 ts_tablet_manager.cc:1398] T 96dcb06388374514951f24393f288eca P 27e3fafad7a24bcda362d2197be87017: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:27.793756 5650 tablet_copy_service.cc:342] P 91e82003a656442189a329b29846a841: Request end of tablet copy session 27e3fafad7a24bcda362d2197be87017-96dcb06388374514951f24393f288eca received from {username='slave'} at 127.3.139.71:34345
I20240502 07:58:27.793859 5650 tablet_copy_service.cc:433] P 91e82003a656442189a329b29846a841: ending tablet copy session 27e3fafad7a24bcda362d2197be87017-96dcb06388374514951f24393f288eca on tablet 96dcb06388374514951f24393f288eca with peer 27e3fafad7a24bcda362d2197be87017
I20240502 07:58:27.795002 5890 raft_consensus.cc:2932] T 5c8c7437c85943ba9113595c17bf6136 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Committing config change with OpId 1.69: config changed from index 67 to 69, VOTER 91e82003a656442189a329b29846a841 (127.3.139.65) evicted. New config: { opid_index: 69 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:27.802660 7333 consensus_queue.cc:1035] T 5c8c7437c85943ba9113595c17bf6136 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 69, Last known committed idx: 67, Time since last communication: 0.000s
I20240502 07:58:27.809823 5759 raft_consensus.cc:1203] T 5c8c7437c85943ba9113595c17bf6136 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.67->[1.68-1.72] Dedup: 1.68->[1.69-1.72]
I20240502 07:58:27.811134 5759 raft_consensus.cc:2932] T 5c8c7437c85943ba9113595c17bf6136 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Committing config change with OpId 1.69: config changed from index 67 to 69, VOTER 91e82003a656442189a329b29846a841 (127.3.139.65) evicted. New config: { opid_index: 69 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:27.826648 7468 tablet_replica.cc:330] T 5c8c7437c85943ba9113595c17bf6136 P 91e82003a656442189a329b29846a841: stopping tablet replica
I20240502 07:58:27.826982 7468 raft_consensus.cc:2229] T 5c8c7437c85943ba9113595c17bf6136 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:27.827205 7468 pending_rounds.cc:62] T 5c8c7437c85943ba9113595c17bf6136 P 91e82003a656442189a329b29846a841: Trying to abort 1 pending ops.
I20240502 07:58:27.827273 7468 pending_rounds.cc:68] T 5c8c7437c85943ba9113595c17bf6136 P 91e82003a656442189a329b29846a841: Aborting op as it isn't in flight: id { term: 1 index: 68 } timestamp: 7023151954584338432 op_type: WRITE_OP write_request { tablet_id: "5c8c7437c85943ba9113595c17bf6136" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\007\000\024\237\257\004|\025\265\356\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000" indirect_data: "<redacted>""81c79f133139329d" } external_consistency_mode: CLIENT_PROPAGATED propagated_timestamp: 7023151954395287552 authz_token { token_data: "\010\313\221\315\261\006\"3\n\005slave\022*\n 28039a31ea894d23af67c1f1da387f21\020\001\030\001 \001(\001" signature: "<redacted>""{\270\001\374g\317\235\353\373\261\005 P\263C\305V\260\216w|\014\226\006n\363\204\262N\272\035t\231V\241\246\266\327\252\242W\245\362\335\264\321\344f\320!V3\027\333\264\005VCu$J7\031\307" signing_key_seq_num: 0 } } request_id { client_id: "287602d09b79464390e8dc120d5fba10" seq_no: 184 first_incomplete_seq_no: 184 attempt_no: 0 }
I20240502 07:58:27.827550 7468 raft_consensus.cc:2258] T 5c8c7437c85943ba9113595c17bf6136 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:27.829288 7468 ts_tablet_manager.cc:1875] T 5c8c7437c85943ba9113595c17bf6136 P 91e82003a656442189a329b29846a841: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:58:27.840751 7468 ts_tablet_manager.cc:1888] T 5c8c7437c85943ba9113595c17bf6136 P 91e82003a656442189a329b29846a841: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.68
I20240502 07:58:27.840871 7468 log.cc:1196] T 5c8c7437c85943ba9113595c17bf6136 P 91e82003a656442189a329b29846a841: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal/wals/5c8c7437c85943ba9113595c17bf6136
I20240502 07:58:27.841408 5473 catalog_manager.cc:4830] TS 91e82003a656442189a329b29846a841 (127.3.139.65:43429): tablet 5c8c7437c85943ba9113595c17bf6136 (table rebalance_test_table_2 [id=28039a31ea894d23af67c1f1da387f21]) successfully deleted
I20240502 07:58:27.849097 7368 raft_consensus.cc:1050] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d: attempting to promote NON_VOTER 27e3fafad7a24bcda362d2197be87017 to VOTER
I20240502 07:58:27.849422 7368 consensus_queue.cc:237] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 63, Committed index: 63, Last appended: 1.63, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 64 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } }
I20240502 07:58:27.851543 7046 raft_consensus.cc:1261] T 5b87554a301044caac956bc5c7f137e8 P 27e3fafad7a24bcda362d2197be87017 [term 1 LEARNER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 63. Preceding OpId from leader: term: 1 index: 64. (index mismatch)
I20240502 07:58:27.854436 5758 raft_consensus.cc:1261] T 5b87554a301044caac956bc5c7f137e8 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 63. Preceding OpId from leader: term: 1 index: 64. (index mismatch)
I20240502 07:58:27.854781 6774 consensus_queue.cc:1035] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 64, Last known committed idx: 62, Time since last communication: 0.000s
I20240502 07:58:27.854941 5890 raft_consensus.cc:1261] T 5b87554a301044caac956bc5c7f137e8 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 63. Preceding OpId from leader: term: 1 index: 64. (index mismatch)
I20240502 07:58:27.855424 6774 consensus_queue.cc:1035] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 64, Last known committed idx: 63, Time since last communication: 0.000s
I20240502 07:58:27.856959 7274 consensus_queue.cc:1035] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 64, Last known committed idx: 63, Time since last communication: 0.000s
I20240502 07:58:27.857187 7193 raft_consensus.cc:1050] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217: attempting to promote NON_VOTER 27e3fafad7a24bcda362d2197be87017 to VOTER
I20240502 07:58:27.857635 7193 consensus_queue.cc:237] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 119, Committed index: 119, Last appended: 1.120, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 121 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } }
I20240502 07:58:27.859364 6284 raft_consensus.cc:1261] T 5b87554a301044caac956bc5c7f137e8 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 63. Preceding OpId from leader: term: 1 index: 64. (index mismatch)
I20240502 07:58:27.860065 6774 consensus_queue.cc:1035] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 64, Last known committed idx: 63, Time since last communication: 0.000s
I20240502 07:58:27.860170 6515 raft_consensus.cc:2932] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d [term 1 LEADER]: Committing config change with OpId 1.64: config changed from index 46 to 64, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 64 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:27.861096 6284 raft_consensus.cc:2932] T 5b87554a301044caac956bc5c7f137e8 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Committing config change with OpId 1.64: config changed from index 46 to 64, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 64 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:27.861541 7046 raft_consensus.cc:1261] T 2a3919f144ed4a27b03e7c4a251499d4 P 27e3fafad7a24bcda362d2197be87017 [term 1 LEARNER]: Refusing update from remote peer e020e7ffe8ce423a8bdefec667161217: Log matching property violated. Preceding OpId in replica: term: 1 index: 120. Preceding OpId from leader: term: 1 index: 121. (index mismatch)
I20240502 07:58:27.863255 7282 consensus_queue.cc:1035] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 121, Last known committed idx: 119, Time since last communication: 0.000s
I20240502 07:58:27.863535 7046 raft_consensus.cc:2932] T 5b87554a301044caac956bc5c7f137e8 P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Committing config change with OpId 1.64: config changed from index 46 to 64, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 64 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:27.863706 6283 raft_consensus.cc:1261] T 2a3919f144ed4a27b03e7c4a251499d4 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Refusing update from remote peer e020e7ffe8ce423a8bdefec667161217: Log matching property violated. Preceding OpId in replica: term: 1 index: 120. Preceding OpId from leader: term: 1 index: 121. (index mismatch)
I20240502 07:58:27.863608 5480 catalog_manager.cc:5484] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d reported cstate change: config changed from index 46 to 64, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d5bc7863cd294f77a932c4a40f98b79d" committed_config { opid_index: 64 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:58:27.864886 7282 consensus_queue.cc:1035] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 121, Last known committed idx: 119, Time since last communication: 0.000s
I20240502 07:58:27.870492 5630 raft_consensus.cc:1261] T 2a3919f144ed4a27b03e7c4a251499d4 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer e020e7ffe8ce423a8bdefec667161217: Log matching property violated. Preceding OpId in replica: term: 1 index: 119. Preceding OpId from leader: term: 1 index: 121. (index mismatch)
I20240502 07:58:27.874418 6021 raft_consensus.cc:1261] T 2a3919f144ed4a27b03e7c4a251499d4 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Refusing update from remote peer e020e7ffe8ce423a8bdefec667161217: Log matching property violated. Preceding OpId in replica: term: 1 index: 120. Preceding OpId from leader: term: 1 index: 121. (index mismatch)
I20240502 07:58:27.874697 7282 consensus_queue.cc:1035] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 121, Last known committed idx: 119, Time since last communication: 0.000s
I20240502 07:58:27.875743 5758 raft_consensus.cc:1261] T 2a3919f144ed4a27b03e7c4a251499d4 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer e020e7ffe8ce423a8bdefec667161217: Log matching property violated. Preceding OpId in replica: term: 1 index: 120. Preceding OpId from leader: term: 1 index: 121. (index mismatch)
I20240502 07:58:27.876061 7282 consensus_queue.cc:1035] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 121, Last known committed idx: 119, Time since last communication: 0.000s
I20240502 07:58:27.878824 7282 raft_consensus.cc:2932] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217 [term 1 LEADER]: Committing config change with OpId 1.121: config changed from index 66 to 121, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 121 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:27.882504 5483 catalog_manager.cc:5484] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217 reported cstate change: config changed from index 66 to 121, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e020e7ffe8ce423a8bdefec667161217" committed_config { opid_index: 121 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:58:27.883224 7048 raft_consensus.cc:2932] T 2a3919f144ed4a27b03e7c4a251499d4 P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Committing config change with OpId 1.121: config changed from index 66 to 121, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 121 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:27.884696 6283 raft_consensus.cc:2932] T 2a3919f144ed4a27b03e7c4a251499d4 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Committing config change with OpId 1.121: config changed from index 66 to 121, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 121 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:27.885732 5759 raft_consensus.cc:2932] T 2a3919f144ed4a27b03e7c4a251499d4 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Committing config change with OpId 1.121: config changed from index 66 to 121, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 121 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:27.892724 7194 consensus_queue.cc:1035] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 121, Last known committed idx: 118, Time since last communication: 0.000s
I20240502 07:58:27.900063 5629 raft_consensus.cc:1261] T 5b87554a301044caac956bc5c7f137e8 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 63. Preceding OpId from leader: term: 1 index: 64. (index mismatch)
I20240502 07:58:27.914700 6020 raft_consensus.cc:2932] T 2a3919f144ed4a27b03e7c4a251499d4 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Committing config change with OpId 1.121: config changed from index 66 to 121, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 121 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:27.919523 5630 raft_consensus.cc:2932] T 2a3919f144ed4a27b03e7c4a251499d4 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Committing config change with OpId 1.121: config changed from index 66 to 121, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 121 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
W20240502 07:58:27.930351 5957 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.139.67:37141, user_credentials={real_user=slave}} blocked reactor thread for 70338.8us
I20240502 07:58:27.932008 5888 raft_consensus.cc:2932] T 5b87554a301044caac956bc5c7f137e8 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Committing config change with OpId 1.64: config changed from index 46 to 64, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 64 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:27.936528 6153 consensus_queue.cc:237] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 122, Committed index: 122, Last appended: 1.123, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 124 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } }
I20240502 07:58:27.941498 5759 raft_consensus.cc:2932] T 5b87554a301044caac956bc5c7f137e8 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Committing config change with OpId 1.64: config changed from index 46 to 64, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 64 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:27.946440 6515 consensus_queue.cc:1035] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 64, Last known committed idx: 62, Time since last communication: 0.000s
I20240502 07:58:27.949323 5629 raft_consensus.cc:1261] T 2a3919f144ed4a27b03e7c4a251499d4 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer e020e7ffe8ce423a8bdefec667161217: Log matching property violated. Preceding OpId in replica: term: 1 index: 121. Preceding OpId from leader: term: 1 index: 124. (index mismatch)
I20240502 07:58:27.954665 5629 raft_consensus.cc:2932] T 5b87554a301044caac956bc5c7f137e8 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Committing config change with OpId 1.64: config changed from index 46 to 64, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 64 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:27.956782 7046 raft_consensus.cc:1261] T 2a3919f144ed4a27b03e7c4a251499d4 P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Refusing update from remote peer e020e7ffe8ce423a8bdefec667161217: Log matching property violated. Preceding OpId in replica: term: 1 index: 122. Preceding OpId from leader: term: 1 index: 124. (index mismatch)
I20240502 07:58:27.956782 6284 raft_consensus.cc:1261] T 2a3919f144ed4a27b03e7c4a251499d4 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Refusing update from remote peer e020e7ffe8ce423a8bdefec667161217: Log matching property violated. Preceding OpId in replica: term: 1 index: 122. Preceding OpId from leader: term: 1 index: 124. (index mismatch)
I20240502 07:58:27.957767 7472 consensus_queue.cc:1035] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 124, Last known committed idx: 122, Time since last communication: 0.000s
I20240502 07:58:27.960347 7472 consensus_queue.cc:1035] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 124, Last known committed idx: 122, Time since last communication: 0.000s
I20240502 07:58:27.967840 7472 consensus_queue.cc:1035] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 124, Last known committed idx: 121, Time since last communication: 0.000s
I20240502 07:58:27.969326 6283 raft_consensus.cc:1203] T 2a3919f144ed4a27b03e7c4a251499d4 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.122->[1.123-1.123] Dedup: 1.123->[]
I20240502 07:58:27.972697 7048 raft_consensus.cc:1203] T 2a3919f144ed4a27b03e7c4a251499d4 P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.122->[1.123-1.124] Dedup: 1.123->[1.124-1.124]
I20240502 07:58:27.973940 7282 raft_consensus.cc:2932] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217 [term 1 LEADER]: Committing config change with OpId 1.124: config changed from index 121 to 124, VOTER eda55a8566c447cfa749457825b70688 (127.3.139.66) evicted. New config: { opid_index: 124 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:27.975936 5474 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 2a3919f144ed4a27b03e7c4a251499d4 with cas_config_opid_index 121: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:58:27.977068 5480 catalog_manager.cc:5484] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217 reported cstate change: config changed from index 121 to 124, VOTER eda55a8566c447cfa749457825b70688 (127.3.139.66) evicted. New cstate: current_term: 1 leader_uuid: "e020e7ffe8ce423a8bdefec667161217" committed_config { opid_index: 124 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:27.980480 5740 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a3919f144ed4a27b03e7c4a251499d4 with delete_type TABLET_DATA_TOMBSTONED (TS eda55a8566c447cfa749457825b70688 not found in new config with opid_index 124) from {username='slave'} at 127.0.0.1:41882
I20240502 07:58:27.984076 7475 tablet_replica.cc:330] T 2a3919f144ed4a27b03e7c4a251499d4 P eda55a8566c447cfa749457825b70688: stopping tablet replica
I20240502 07:58:27.984925 7475 raft_consensus.cc:2229] T 2a3919f144ed4a27b03e7c4a251499d4 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:27.985402 7475 pending_rounds.cc:62] T 2a3919f144ed4a27b03e7c4a251499d4 P eda55a8566c447cfa749457825b70688: Trying to abort 1 pending ops.
I20240502 07:58:27.985474 7475 pending_rounds.cc:68] T 2a3919f144ed4a27b03e7c4a251499d4 P eda55a8566c447cfa749457825b70688: Aborting op as it isn't in flight: id { term: 1 index: 123 } timestamp: 7023151955664035840 op_type: WRITE_OP write_request { tablet_id: "2a3919f144ed4a27b03e7c4a251499d4" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\007\000C\003\315Y\226,\217r\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000" indirect_data: "<redacted>""3eac4b73c31b57fb" } external_consistency_mode: CLIENT_PROPAGATED propagated_timestamp: 7023151955623014400 authz_token { token_data: "\010\313\221\315\261\006\"3\n\005slave\022*\n d0a0bf64867c4915ad797ab2bcd7101f\020\001\030\001 \001(\001" signature: "<redacted>""\271\255\033.\277\314\302\260\006\013\236@\273u\027\224\264\203\227&\204}\313Px\014\371M5\271\346\343\334\024k\270\304\236q\250V\225UW\212b\326\363 \211\243\314\214\261\236\004\365\240\330\376\371\352\226\033" signing_key_seq_num: 0 } } request_id { client_id: "05cd43938067401cb649a0c9aede8230" seq_no: 369 first_incomplete_seq_no: 369 attempt_no: 0 }
I20240502 07:58:27.986863 7048 raft_consensus.cc:2932] T 2a3919f144ed4a27b03e7c4a251499d4 P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Committing config change with OpId 1.124: config changed from index 121 to 124, VOTER eda55a8566c447cfa749457825b70688 (127.3.139.66) evicted. New config: { opid_index: 124 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:27.987905 6022 raft_consensus.cc:1261] T 2a3919f144ed4a27b03e7c4a251499d4 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Refusing update from remote peer e020e7ffe8ce423a8bdefec667161217: Log matching property violated. Preceding OpId in replica: term: 1 index: 123. Preceding OpId from leader: term: 1 index: 124. (index mismatch)
I20240502 07:58:27.988471 7472 consensus_queue.cc:1035] T 2a3919f144ed4a27b03e7c4a251499d4 P e020e7ffe8ce423a8bdefec667161217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 124, Last known committed idx: 122, Time since last communication: 0.000s
I20240502 07:58:27.988612 6284 raft_consensus.cc:2932] T 2a3919f144ed4a27b03e7c4a251499d4 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Committing config change with OpId 1.124: config changed from index 121 to 124, VOTER eda55a8566c447cfa749457825b70688 (127.3.139.66) evicted. New config: { opid_index: 124 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:27.985811 7475 raft_consensus.cc:2258] T 2a3919f144ed4a27b03e7c4a251499d4 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:27.990123 7475 ts_tablet_manager.cc:1875] T 2a3919f144ed4a27b03e7c4a251499d4 P eda55a8566c447cfa749457825b70688: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:58:27.990655 6022 raft_consensus.cc:2932] T 2a3919f144ed4a27b03e7c4a251499d4 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Committing config change with OpId 1.124: config changed from index 121 to 124, VOTER eda55a8566c447cfa749457825b70688 (127.3.139.66) evicted. New config: { opid_index: 124 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:27.994993 7475 ts_tablet_manager.cc:1888] T 2a3919f144ed4a27b03e7c4a251499d4 P eda55a8566c447cfa749457825b70688: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.123
I20240502 07:58:27.995098 7475 log.cc:1196] T 2a3919f144ed4a27b03e7c4a251499d4 P eda55a8566c447cfa749457825b70688: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal/wals/2a3919f144ed4a27b03e7c4a251499d4
I20240502 07:58:27.995610 5473 catalog_manager.cc:4830] TS eda55a8566c447cfa749457825b70688 (127.3.139.66:39925): tablet 2a3919f144ed4a27b03e7c4a251499d4 (table rebalance_test_table_3 [id=d0a0bf64867c4915ad797ab2bcd7101f]) successfully deleted
I20240502 07:58:27.996632 5630 raft_consensus.cc:1203] T 2a3919f144ed4a27b03e7c4a251499d4 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.121->[1.122-1.124] Dedup: 1.123->[1.124-1.124]
I20240502 07:58:28.025152 7046 raft_consensus.cc:1203] T 96dcb06388374514951f24393f288eca P 27e3fafad7a24bcda362d2197be87017 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.82->[1.83-1.126] Dedup: 1.112->[1.113-1.126]
I20240502 07:58:28.049657 5629 raft_consensus.cc:2932] T 2a3919f144ed4a27b03e7c4a251499d4 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Committing config change with OpId 1.124: config changed from index 121 to 124, VOTER eda55a8566c447cfa749457825b70688 (127.3.139.66) evicted. New config: { opid_index: 124 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:28.086457 6023 consensus_queue.cc:237] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 65, Committed index: 65, Last appended: 1.65, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } }
I20240502 07:58:28.088944 6284 raft_consensus.cc:1261] T 5b87554a301044caac956bc5c7f137e8 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 65. Preceding OpId from leader: term: 1 index: 66. (index mismatch)
I20240502 07:58:28.089291 6515 consensus_queue.cc:1035] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 66, Last known committed idx: 65, Time since last communication: 0.000s
I20240502 07:58:28.091498 7046 raft_consensus.cc:1261] T 5b87554a301044caac956bc5c7f137e8 P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 65. Preceding OpId from leader: term: 1 index: 66. (index mismatch)
I20240502 07:58:28.092465 7368 consensus_queue.cc:1035] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 66, Last known committed idx: 64, Time since last communication: 0.000s
I20240502 07:58:28.094739 7368 raft_consensus.cc:2932] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d [term 1 LEADER]: Committing config change with OpId 1.66: config changed from index 64 to 66, VOTER 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67) evicted. New config: { opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:28.097193 7046 raft_consensus.cc:2932] T 5b87554a301044caac956bc5c7f137e8 P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Committing config change with OpId 1.66: config changed from index 64 to 66, VOTER 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67) evicted. New config: { opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:28.098124 5480 catalog_manager.cc:5484] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d reported cstate change: config changed from index 64 to 66, VOTER 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67) evicted. New cstate: current_term: 1 leader_uuid: "d5bc7863cd294f77a932c4a40f98b79d" committed_config { opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:58:28.100206 5473 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 5b87554a301044caac956bc5c7f137e8 with cas_config_opid_index 64: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:58:28.101023 6283 raft_consensus.cc:2932] T 5b87554a301044caac956bc5c7f137e8 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Committing config change with OpId 1.66: config changed from index 64 to 66, VOTER 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67) evicted. New config: { opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:28.102123 5757 raft_consensus.cc:1261] T 5b87554a301044caac956bc5c7f137e8 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 65. Preceding OpId from leader: term: 1 index: 66. (index mismatch)
I20240502 07:58:28.102427 5872 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b87554a301044caac956bc5c7f137e8 with delete_type TABLET_DATA_TOMBSTONED (TS 89bf5c0a6b2a4d949851089446f17b96 not found in new config with opid_index 66) from {username='slave'} at 127.0.0.1:42126
I20240502 07:58:28.103674 7368 consensus_queue.cc:1035] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 66, Last known committed idx: 64, Time since last communication: 0.000s
I20240502 07:58:28.104416 7334 raft_consensus.cc:1050] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841: attempting to promote NON_VOTER 27e3fafad7a24bcda362d2197be87017 to VOTER
I20240502 07:58:28.104831 7481 tablet_replica.cc:330] T 5b87554a301044caac956bc5c7f137e8 P 89bf5c0a6b2a4d949851089446f17b96: stopping tablet replica
I20240502 07:58:28.104864 7334 consensus_queue.cc:237] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 126, Committed index: 126, Last appended: 1.127, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 128 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } }
I20240502 07:58:28.105063 7481 raft_consensus.cc:2229] T 5b87554a301044caac956bc5c7f137e8 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:28.105188 7481 raft_consensus.cc:2258] T 5b87554a301044caac956bc5c7f137e8 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:28.106388 6284 raft_consensus.cc:1261] T 96dcb06388374514951f24393f288eca P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 127. Preceding OpId from leader: term: 1 index: 128. (index mismatch)
I20240502 07:58:28.106400 7481 ts_tablet_manager.cc:1875] T 5b87554a301044caac956bc5c7f137e8 P 89bf5c0a6b2a4d949851089446f17b96: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:58:28.106547 6150 raft_consensus.cc:1261] T 96dcb06388374514951f24393f288eca P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 127. Preceding OpId from leader: term: 1 index: 128. (index mismatch)
I20240502 07:58:28.106772 7326 consensus_queue.cc:1035] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 128, Last known committed idx: 126, Time since last communication: 0.000s
I20240502 07:58:28.107244 7048 raft_consensus.cc:1261] T 96dcb06388374514951f24393f288eca P 27e3fafad7a24bcda362d2197be87017 [term 1 LEARNER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 127. Preceding OpId from leader: term: 1 index: 128. (index mismatch)
I20240502 07:58:28.107226 7326 consensus_queue.cc:1035] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 128, Last known committed idx: 126, Time since last communication: 0.000s
I20240502 07:58:28.107739 6770 consensus_queue.cc:1035] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 128, Last known committed idx: 126, Time since last communication: 0.000s
I20240502 07:58:28.108435 5628 raft_consensus.cc:1261] T 5b87554a301044caac956bc5c7f137e8 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 65. Preceding OpId from leader: term: 1 index: 66. (index mismatch)
I20240502 07:58:28.109840 6023 raft_consensus.cc:1261] T 96dcb06388374514951f24393f288eca P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 126. Preceding OpId from leader: term: 1 index: 128. (index mismatch)
I20240502 07:58:28.110747 7368 consensus_queue.cc:1035] T 5b87554a301044caac956bc5c7f137e8 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 66, Last known committed idx: 65, Time since last communication: 0.000s
I20240502 07:58:28.111202 5757 raft_consensus.cc:1261] T 96dcb06388374514951f24393f288eca P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 126. Preceding OpId from leader: term: 1 index: 128. (index mismatch)
I20240502 07:58:28.111249 7379 tablet_bootstrap.cc:492] T d41a2cb7b69e40d9a681384e5ad0a451 P 27e3fafad7a24bcda362d2197be87017: Bootstrap replayed 1/1 log segments. Stats: ops{read=50 overwritten=0 applied=49 ignored=0} inserts{seen=734 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240502 07:58:28.111676 7379 tablet_bootstrap.cc:492] T d41a2cb7b69e40d9a681384e5ad0a451 P 27e3fafad7a24bcda362d2197be87017: Bootstrap complete.
I20240502 07:58:28.111919 7379 ts_tablet_manager.cc:1367] T d41a2cb7b69e40d9a681384e5ad0a451 P 27e3fafad7a24bcda362d2197be87017: Time spent bootstrapping tablet: real 0.935s user 0.056s sys 0.003s
I20240502 07:58:28.112287 5759 raft_consensus.cc:2932] T 5b87554a301044caac956bc5c7f137e8 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Committing config change with OpId 1.66: config changed from index 64 to 66, VOTER 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67) evicted. New config: { opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:28.112658 7379 raft_consensus.cc:348] T d41a2cb7b69e40d9a681384e5ad0a451 P 27e3fafad7a24bcda362d2197be87017 [term 1 LEARNER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } }
I20240502 07:58:28.113193 7326 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [term 1 LEADER]: Committing config change with OpId 1.128: config changed from index 83 to 128, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 128 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:28.113389 6770 consensus_queue.cc:1035] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 128, Last known committed idx: 126, Time since last communication: 0.000s
I20240502 07:58:28.113952 6770 consensus_queue.cc:1035] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 128, Last known committed idx: 126, Time since last communication: 0.000s
I20240502 07:58:28.114813 7481 ts_tablet_manager.cc:1888] T 5b87554a301044caac956bc5c7f137e8 P 89bf5c0a6b2a4d949851089446f17b96: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.65
I20240502 07:58:28.115453 6022 raft_consensus.cc:1203] T 96dcb06388374514951f24393f288eca P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.126->[1.127-1.128] Dedup: 1.127->[1.128-1.128]
I20240502 07:58:28.115870 6022 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Committing config change with OpId 1.128: config changed from index 83 to 128, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 128 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:28.117446 5630 raft_consensus.cc:2932] T 5b87554a301044caac956bc5c7f137e8 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Committing config change with OpId 1.66: config changed from index 64 to 66, VOTER 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67) evicted. New config: { opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:28.117911 7379 raft_consensus.cc:729] T d41a2cb7b69e40d9a681384e5ad0a451 P 27e3fafad7a24bcda362d2197be87017 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 27e3fafad7a24bcda362d2197be87017, State: Initialized, Role: LEARNER
I20240502 07:58:28.118064 7379 consensus_queue.cc:260] T d41a2cb7b69e40d9a681384e5ad0a451 P 27e3fafad7a24bcda362d2197be87017 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 49, Last appended: 1.50, Last appended by leader: 50, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } }
I20240502 07:58:28.118417 7379 ts_tablet_manager.cc:1398] T d41a2cb7b69e40d9a681384e5ad0a451 P 27e3fafad7a24bcda362d2197be87017: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20240502 07:58:28.122938 7046 raft_consensus.cc:1203] T d41a2cb7b69e40d9a681384e5ad0a451 P 27e3fafad7a24bcda362d2197be87017 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.44->[1.45-1.62] Dedup: 1.50->[1.51-1.62]
I20240502 07:58:28.125370 5484 catalog_manager.cc:5484] T 96dcb06388374514951f24393f288eca P d5bc7863cd294f77a932c4a40f98b79d reported cstate change: config changed from index 83 to 128, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "91e82003a656442189a329b29846a841" committed_config { opid_index: 128 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:28.137162 6282 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Committing config change with OpId 1.128: config changed from index 83 to 128, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 128 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:28.139441 6303 tablet_copy_service.cc:342] P d58f6d1ff8cf472f8a9edf3700e64bfb: Request end of tablet copy session 27e3fafad7a24bcda362d2197be87017-d41a2cb7b69e40d9a681384e5ad0a451 received from {username='slave'} at 127.3.139.71:33001
I20240502 07:58:28.139531 6303 tablet_copy_service.cc:433] P d58f6d1ff8cf472f8a9edf3700e64bfb: ending tablet copy session 27e3fafad7a24bcda362d2197be87017-d41a2cb7b69e40d9a681384e5ad0a451 on tablet d41a2cb7b69e40d9a681384e5ad0a451 with peer 27e3fafad7a24bcda362d2197be87017
I20240502 07:58:28.147650 5758 raft_consensus.cc:1203] T 96dcb06388374514951f24393f288eca P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.126->[1.127-1.128] Dedup: 1.127->[1.128-1.128]
I20240502 07:58:28.148291 6152 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.128: config changed from index 83 to 128, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 128 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:28.149171 7048 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Committing config change with OpId 1.128: config changed from index 83 to 128, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 128 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:28.171782 7219 raft_consensus.cc:1050] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb: attempting to promote NON_VOTER 27e3fafad7a24bcda362d2197be87017 to VOTER
I20240502 07:58:28.172224 7219 consensus_queue.cc:237] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 63, Committed index: 63, Last appended: 1.63, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 64 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } }
I20240502 07:58:28.173434 7046 raft_consensus.cc:1261] T d41a2cb7b69e40d9a681384e5ad0a451 P 27e3fafad7a24bcda362d2197be87017 [term 1 LEARNER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 63. Preceding OpId from leader: term: 1 index: 64. (index mismatch)
I20240502 07:58:28.173854 7253 consensus_queue.cc:1035] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 64, Last known committed idx: 63, Time since last communication: 0.000s
I20240502 07:58:28.174795 6153 raft_consensus.cc:1261] T d41a2cb7b69e40d9a681384e5ad0a451 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 63. Preceding OpId from leader: term: 1 index: 64. (index mismatch)
I20240502 07:58:28.176164 5629 raft_consensus.cc:1261] T d41a2cb7b69e40d9a681384e5ad0a451 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 63. Preceding OpId from leader: term: 1 index: 64. (index mismatch)
I20240502 07:58:28.176595 7308 consensus_queue.cc:1035] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 64, Last known committed idx: 63, Time since last communication: 0.000s
I20240502 07:58:28.176735 7253 consensus_queue.cc:1035] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 64, Last known committed idx: 63, Time since last communication: 0.000s
I20240502 07:58:28.181149 6928 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 LEADER]: Committing config change with OpId 1.64: config changed from index 45 to 64, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 64 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:28.183326 7046 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Committing config change with OpId 1.64: config changed from index 45 to 64, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 64 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:28.183868 5629 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Committing config change with OpId 1.64: config changed from index 45 to 64, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 64 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:28.183988 6149 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.64: config changed from index 45 to 64, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 64 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:28.186213 5484 catalog_manager.cc:5484] T d41a2cb7b69e40d9a681384e5ad0a451 P 91e82003a656442189a329b29846a841 reported cstate change: config changed from index 45 to 64, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" committed_config { opid_index: 64 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:28.190860 6023 raft_consensus.cc:1261] T d41a2cb7b69e40d9a681384e5ad0a451 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 63. Preceding OpId from leader: term: 1 index: 64. (index mismatch)
I20240502 07:58:28.193025 5758 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Committing config change with OpId 1.128: config changed from index 83 to 128, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 128 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:28.196595 7481 log.cc:1196] T 5b87554a301044caac956bc5c7f137e8 P 89bf5c0a6b2a4d949851089446f17b96: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal/wals/5b87554a301044caac956bc5c7f137e8
I20240502 07:58:28.179389 5888 raft_consensus.cc:1261] T d41a2cb7b69e40d9a681384e5ad0a451 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 63. Preceding OpId from leader: term: 1 index: 64. (index mismatch)
I20240502 07:58:28.197378 5473 catalog_manager.cc:4830] TS 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67:37141): tablet 5b87554a301044caac956bc5c7f137e8 (table rebalance_test_table_0 [id=e1e32c7b8b8c4c05b2f82c3195606811]) successfully deleted
I20240502 07:58:28.204844 7253 consensus_queue.cc:1035] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 64, Last known committed idx: 63, Time since last communication: 0.000s
I20240502 07:58:28.205843 7219 consensus_queue.cc:1035] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 64, Last known committed idx: 63, Time since last communication: 0.000s
I20240502 07:58:28.206686 6021 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Committing config change with OpId 1.64: config changed from index 45 to 64, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 64 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:28.217170 5629 consensus_queue.cc:237] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 131, Committed index: 131, Last appended: 1.132, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 133 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } }
I20240502 07:58:28.220827 7048 raft_consensus.cc:1261] T 96dcb06388374514951f24393f288eca P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 132. Preceding OpId from leader: term: 1 index: 133. (index mismatch)
I20240502 07:58:28.221750 5888 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Committing config change with OpId 1.64: config changed from index 45 to 64, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 64 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:28.224953 6023 raft_consensus.cc:1261] T 96dcb06388374514951f24393f288eca P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 131. Preceding OpId from leader: term: 1 index: 133. (index mismatch)
I20240502 07:58:28.225739 5757 raft_consensus.cc:1261] T 96dcb06388374514951f24393f288eca P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 132. Preceding OpId from leader: term: 1 index: 133. (index mismatch)
I20240502 07:58:28.226642 7326 consensus_queue.cc:1035] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 133, Last known committed idx: 131, Time since last communication: 0.000s
I20240502 07:58:28.231631 6770 consensus_queue.cc:1035] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 133, Last known committed idx: 131, Time since last communication: 0.000s
I20240502 07:58:28.233913 6770 consensus_queue.cc:1035] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 133, Last known committed idx: 131, Time since last communication: 0.000s
I20240502 07:58:28.243085 6285 consensus_queue.cc:237] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 66, Committed index: 66, Last appended: 1.66, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 67 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } }
I20240502 07:58:28.244390 7046 raft_consensus.cc:1261] T d41a2cb7b69e40d9a681384e5ad0a451 P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 66. Preceding OpId from leader: term: 1 index: 67. (index mismatch)
I20240502 07:58:28.244961 7219 consensus_queue.cc:1035] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 67, Last known committed idx: 66, Time since last communication: 0.000s
I20240502 07:58:28.251148 6149 raft_consensus.cc:1261] T 96dcb06388374514951f24393f288eca P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 131. Preceding OpId from leader: term: 1 index: 133. (index mismatch)
I20240502 07:58:28.252351 6153 raft_consensus.cc:1261] T d41a2cb7b69e40d9a681384e5ad0a451 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 66. Preceding OpId from leader: term: 1 index: 67. (index mismatch)
I20240502 07:58:28.253367 7253 consensus_queue.cc:1035] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 67, Last known committed idx: 66, Time since last communication: 0.000s
I20240502 07:58:28.254317 7197 consensus_queue.cc:1035] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 133, Last known committed idx: 130, Time since last communication: 0.000s
I20240502 07:58:28.255806 7253 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 LEADER]: Committing config change with OpId 1.67: config changed from index 64 to 67, VOTER d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68) evicted. New config: { opid_index: 67 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:28.256088 6150 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.67: config changed from index 64 to 67, VOTER d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68) evicted. New config: { opid_index: 67 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:28.257197 5474 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet d41a2cb7b69e40d9a681384e5ad0a451 with cas_config_opid_index 64: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:58:28.257490 7048 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Committing config change with OpId 1.67: config changed from index 64 to 67, VOTER d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68) evicted. New config: { opid_index: 67 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:28.257930 5484 catalog_manager.cc:5484] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb reported cstate change: config changed from index 64 to 67, VOTER d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68) evicted. New cstate: current_term: 1 leader_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" committed_config { opid_index: 67 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:58:28.264218 6002 tablet_service.cc:1514] Processing DeleteTablet for tablet d41a2cb7b69e40d9a681384e5ad0a451 with delete_type TABLET_DATA_TOMBSTONED (TS d5bc7863cd294f77a932c4a40f98b79d not found in new config with opid_index 67) from {username='slave'} at 127.0.0.1:39270
I20240502 07:58:28.264580 5892 raft_consensus.cc:1261] T d41a2cb7b69e40d9a681384e5ad0a451 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 66. Preceding OpId from leader: term: 1 index: 67. (index mismatch)
I20240502 07:58:28.266264 5626 raft_consensus.cc:1261] T d41a2cb7b69e40d9a681384e5ad0a451 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 66. Preceding OpId from leader: term: 1 index: 67. (index mismatch)
I20240502 07:58:28.270879 7482 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [term 1 LEADER]: Committing config change with OpId 1.133: config changed from index 128 to 133, VOTER d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70) evicted. New config: { opid_index: 133 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:28.274145 6149 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.133: config changed from index 128 to 133, VOTER d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70) evicted. New config: { opid_index: 133 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:28.276686 5473 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 96dcb06388374514951f24393f288eca with cas_config_opid_index 128: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:58:28.277171 5480 catalog_manager.cc:5484] T 96dcb06388374514951f24393f288eca P e020e7ffe8ce423a8bdefec667161217 reported cstate change: config changed from index 128 to 133, VOTER d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70) evicted. New cstate: current_term: 1 leader_uuid: "91e82003a656442189a329b29846a841" committed_config { opid_index: 133 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:28.280187 6263 tablet_service.cc:1514] Processing DeleteTablet for tablet 96dcb06388374514951f24393f288eca with delete_type TABLET_DATA_TOMBSTONED (TS d58f6d1ff8cf472f8a9edf3700e64bfb not found in new config with opid_index 133) from {username='slave'} at 127.0.0.1:33086
I20240502 07:58:28.285727 7048 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Committing config change with OpId 1.133: config changed from index 128 to 133, VOTER d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70) evicted. New config: { opid_index: 133 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:28.287494 7219 consensus_queue.cc:1035] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 67, Last known committed idx: 66, Time since last communication: 0.000s
I20240502 07:58:28.289372 5756 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Committing config change with OpId 1.133: config changed from index 128 to 133, VOTER d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70) evicted. New config: { opid_index: 133 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:28.290673 7492 tablet_replica.cc:330] T d41a2cb7b69e40d9a681384e5ad0a451 P d5bc7863cd294f77a932c4a40f98b79d: stopping tablet replica
I20240502 07:58:28.290920 6928 consensus_queue.cc:1035] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 67, Last known committed idx: 66, Time since last communication: 0.000s
I20240502 07:58:28.290974 7492 raft_consensus.cc:2229] T d41a2cb7b69e40d9a681384e5ad0a451 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:28.292867 7492 pending_rounds.cc:62] T d41a2cb7b69e40d9a681384e5ad0a451 P d5bc7863cd294f77a932c4a40f98b79d: Trying to abort 1 pending ops.
I20240502 07:58:28.292961 7492 pending_rounds.cc:68] T d41a2cb7b69e40d9a681384e5ad0a451 P d5bc7863cd294f77a932c4a40f98b79d: Aborting op as it isn't in flight: id { term: 1 index: 66 } timestamp: 7023151956914499584 op_type: WRITE_OP write_request { tablet_id: "d41a2cb7b69e40d9a681384e5ad0a451" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\007\000\327tdM\230\312\362C\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000" indirect_data: "<redacted>""2ebb9dcfbeeb73a9" } external_consistency_mode: CLIENT_PROPAGATED propagated_timestamp: 7023151956843171840 authz_token { token_data: "\010\313\221\315\261\006\"3\n\005slave\022*\n 6f5d44f6fbe544099b33ad94e35cee6e\020\001\030\001 \001(\001" signature: "<redacted>""w\242UF\371,\371\340\013\017\266\270\032i\254\374+9\235\'JV\013\334\205\031;(\005|\330|\240\353(nm\260\362\336\221\251n\254\2773u\301K\350\006\375\243\356\322*FW\324\272G\350\204\273" signing_key_seq_num: 0 } } request_id { client_id: "85d95aa7aaf24d5d9156d56c8068ca7a" seq_no: 205 first_incomplete_seq_no: 205 attempt_no: 0 }
I20240502 07:58:28.293248 7492 raft_consensus.cc:2258] T d41a2cb7b69e40d9a681384e5ad0a451 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:28.294992 7492 ts_tablet_manager.cc:1875] T d41a2cb7b69e40d9a681384e5ad0a451 P d5bc7863cd294f77a932c4a40f98b79d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:58:28.296566 5892 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Committing config change with OpId 1.67: config changed from index 64 to 67, VOTER d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68) evicted. New config: { opid_index: 67 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:28.297394 7492 ts_tablet_manager.cc:1888] T d41a2cb7b69e40d9a681384e5ad0a451 P d5bc7863cd294f77a932c4a40f98b79d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.66
I20240502 07:58:28.297468 7492 log.cc:1196] T d41a2cb7b69e40d9a681384e5ad0a451 P d5bc7863cd294f77a932c4a40f98b79d: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal/wals/d41a2cb7b69e40d9a681384e5ad0a451
I20240502 07:58:28.298077 5473 catalog_manager.cc:4830] TS d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68:44287): tablet d41a2cb7b69e40d9a681384e5ad0a451 (table rebalance_test_table_1 [id=6f5d44f6fbe544099b33ad94e35cee6e]) successfully deleted
I20240502 07:58:28.299276 5626 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Committing config change with OpId 1.67: config changed from index 64 to 67, VOTER d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68) evicted. New config: { opid_index: 67 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:28.303606 6022 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Committing config change with OpId 1.133: config changed from index 128 to 133, VOTER d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70) evicted. New config: { opid_index: 133 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:28.332744 7493 tablet_replica.cc:330] T 96dcb06388374514951f24393f288eca P d58f6d1ff8cf472f8a9edf3700e64bfb: stopping tablet replica
I20240502 07:58:28.337055 7493 raft_consensus.cc:2229] T 96dcb06388374514951f24393f288eca P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:28.342952 7493 pending_rounds.cc:62] T 96dcb06388374514951f24393f288eca P d58f6d1ff8cf472f8a9edf3700e64bfb: Trying to abort 1 pending ops.
I20240502 07:58:28.355759 7493 pending_rounds.cc:68] T 96dcb06388374514951f24393f288eca P d58f6d1ff8cf472f8a9edf3700e64bfb: Aborting op as it isn't in flight: id { term: 1 index: 132 } timestamp: 7023151956826259456 op_type: WRITE_OP write_request { tablet_id: "96dcb06388374514951f24393f288eca" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\007\000\313GK\027\252\371\344\262\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000" indirect_data: "<redacted>""1276ad405ad09c70" } external_consistency_mode: CLIENT_PROPAGATED propagated_timestamp: 7023151956767096832 authz_token { token_data: "\010\313\221\315\261\006\"3\n\005slave\022*\n d0a0bf64867c4915ad797ab2bcd7101f\020\001\030\001 \001(\001" signature: "<redacted>""\271\255\033.\277\314\302\260\006\013\236@\273u\027\224\264\203\227&\204}\313Px\014\371M5\271\346\343\334\024k\270\304\236q\250V\225UW\212b\326\363 \211\243\314\214\261\236\004\365\240\330\376\371\352\226\033" signing_key_seq_num: 0 } } request_id { client_id: "05cd43938067401cb649a0c9aede8230" seq_no: 382 first_incomplete_seq_no: 382 attempt_no: 0 }
I20240502 07:58:28.364149 7493 raft_consensus.cc:2258] T 96dcb06388374514951f24393f288eca P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:28.373309 7493 ts_tablet_manager.cc:1875] T 96dcb06388374514951f24393f288eca P d58f6d1ff8cf472f8a9edf3700e64bfb: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:58:28.482779 7493 ts_tablet_manager.cc:1888] T 96dcb06388374514951f24393f288eca P d58f6d1ff8cf472f8a9edf3700e64bfb: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.132
I20240502 07:58:28.487185 7493 log.cc:1196] T 96dcb06388374514951f24393f288eca P d58f6d1ff8cf472f8a9edf3700e64bfb: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/wal/wals/96dcb06388374514951f24393f288eca
I20240502 07:58:28.506589 5474 catalog_manager.cc:4830] TS d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70:41047): tablet 96dcb06388374514951f24393f288eca (table rebalance_test_table_3 [id=d0a0bf64867c4915ad797ab2bcd7101f]) successfully deleted
I20240502 07:58:28.850499 6285 consensus_queue.cc:237] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 81, Committed index: 81, Last appended: 1.81, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 82 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } }
I20240502 07:58:28.853893 6153 raft_consensus.cc:1261] T 7001a52697f1417bb84a0e3ed673ed0c P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 81. Preceding OpId from leader: term: 1 index: 82. (index mismatch)
I20240502 07:58:28.854791 5629 raft_consensus.cc:1261] T 7001a52697f1417bb84a0e3ed673ed0c P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 81. Preceding OpId from leader: term: 1 index: 82. (index mismatch)
I20240502 07:58:28.855641 6021 raft_consensus.cc:1261] T 7001a52697f1417bb84a0e3ed673ed0c P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 81. Preceding OpId from leader: term: 1 index: 82. (index mismatch)
I20240502 07:58:28.856899 6928 consensus_queue.cc:1035] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 82, Last known committed idx: 81, Time since last communication: 0.000s
I20240502 07:58:28.857726 7212 consensus_queue.cc:1035] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 82, Last known committed idx: 80, Time since last communication: 0.000s
I20240502 07:58:28.860556 6928 consensus_queue.cc:1035] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 82, Last known committed idx: 81, Time since last communication: 0.000s
W20240502 07:58:28.862558 6220 consensus_peers.cc:485] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb -> Peer 27e3fafad7a24bcda362d2197be87017 (127.3.139.71:35341): Couldn't send request to peer 27e3fafad7a24bcda362d2197be87017. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7001a52697f1417bb84a0e3ed673ed0c. This is attempt 1: this message will repeat every 5th retry.
W20240502 07:58:28.863569 5564 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.139.66:39925, user_credentials={real_user=slave}} blocked reactor thread for 81346.4us
I20240502 07:58:28.866775 7254 raft_consensus.cc:2932] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 LEADER]: Committing config change with OpId 1.82: config changed from index -1 to 82, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 82 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:28.869777 5480 catalog_manager.cc:5484] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb reported cstate change: config changed from index -1 to 82, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New cstate: current_term: 1 leader_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" committed_config { opid_index: 82 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:28.871687 6021 raft_consensus.cc:2932] T 7001a52697f1417bb84a0e3ed673ed0c P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Committing config change with OpId 1.82: config changed from index -1 to 82, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 82 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:28.872395 5892 raft_consensus.cc:1261] T 7001a52697f1417bb84a0e3ed673ed0c P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 81. Preceding OpId from leader: term: 1 index: 82. (index mismatch)
I20240502 07:58:28.874126 6928 consensus_queue.cc:1035] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 82, Last known committed idx: 80, Time since last communication: 0.000s
I20240502 07:58:28.875715 5892 raft_consensus.cc:2932] T 7001a52697f1417bb84a0e3ed673ed0c P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Committing config change with OpId 1.82: config changed from index -1 to 82, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 82 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:28.900408 6153 raft_consensus.cc:2932] T 7001a52697f1417bb84a0e3ed673ed0c P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.82: config changed from index -1 to 82, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 82 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:28.869668 5629 raft_consensus.cc:2932] T 7001a52697f1417bb84a0e3ed673ed0c P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Committing config change with OpId 1.82: config changed from index -1 to 82, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 82 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:29.111385 6283 consensus_queue.cc:237] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 44, Committed index: 44, Last appended: 1.44, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } }
W20240502 07:58:29.113005 6220 consensus_peers.cc:485] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb -> Peer 27e3fafad7a24bcda362d2197be87017 (127.3.139.71:35341): Couldn't send request to peer 27e3fafad7a24bcda362d2197be87017. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7f3d5d24bc164883adb1a11552500ce6. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:29.115586 6020 raft_consensus.cc:1261] T 7f3d5d24bc164883adb1a11552500ce6 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 44. Preceding OpId from leader: term: 1 index: 45. (index mismatch)
I20240502 07:58:29.115725 5889 raft_consensus.cc:1261] T 7f3d5d24bc164883adb1a11552500ce6 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 44. Preceding OpId from leader: term: 1 index: 45. (index mismatch)
I20240502 07:58:29.115782 5759 raft_consensus.cc:1261] T 7f3d5d24bc164883adb1a11552500ce6 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 44. Preceding OpId from leader: term: 1 index: 45. (index mismatch)
I20240502 07:58:29.118335 7253 consensus_queue.cc:1035] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 45, Last known committed idx: 44, Time since last communication: 0.000s
I20240502 07:58:29.120196 6674 consensus_queue.cc:1035] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 45, Last known committed idx: 44, Time since last communication: 0.000s
I20240502 07:58:29.121253 6674 consensus_queue.cc:1035] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 45, Last known committed idx: 44, Time since last communication: 0.000s
I20240502 07:58:29.125844 5626 raft_consensus.cc:1261] T 7f3d5d24bc164883adb1a11552500ce6 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 44. Preceding OpId from leader: term: 1 index: 45. (index mismatch)
I20240502 07:58:29.153474 6674 raft_consensus.cc:2932] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 LEADER]: Committing config change with OpId 1.45: config changed from index -1 to 45, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:29.155380 7219 consensus_queue.cc:1035] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 45, Last known committed idx: 44, Time since last communication: 0.000s
I20240502 07:58:29.160270 5483 catalog_manager.cc:5484] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb reported cstate change: config changed from index -1 to 45, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New cstate: current_term: 1 leader_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" committed_config { opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:29.174325 5759 raft_consensus.cc:2932] T 7f3d5d24bc164883adb1a11552500ce6 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Committing config change with OpId 1.45: config changed from index -1 to 45, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:29.178679 6019 raft_consensus.cc:2932] T 7f3d5d24bc164883adb1a11552500ce6 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Committing config change with OpId 1.45: config changed from index -1 to 45, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:29.182999 5626 raft_consensus.cc:2932] T 7f3d5d24bc164883adb1a11552500ce6 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Committing config change with OpId 1.45: config changed from index -1 to 45, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:29.207885 5628 consensus_queue.cc:237] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 84, Committed index: 84, Last appended: 1.85, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 86 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } }
W20240502 07:58:29.209628 5565 consensus_peers.cc:485] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 -> Peer 27e3fafad7a24bcda362d2197be87017 (127.3.139.71:35341): Couldn't send request to peer 27e3fafad7a24bcda362d2197be87017. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 18a448dbe2af493abefbc728a8f85186. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:29.209719 5892 raft_consensus.cc:1261] T 18a448dbe2af493abefbc728a8f85186 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 85. Preceding OpId from leader: term: 1 index: 86. (index mismatch)
I20240502 07:58:29.210352 6740 consensus_queue.cc:1035] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 86, Last known committed idx: 84, Time since last communication: 0.000s
I20240502 07:58:29.212363 6020 raft_consensus.cc:1261] T 18a448dbe2af493abefbc728a8f85186 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 84. Preceding OpId from leader: term: 1 index: 86. (index mismatch)
I20240502 07:58:29.212321 5892 raft_consensus.cc:2932] T 7f3d5d24bc164883adb1a11552500ce6 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Committing config change with OpId 1.45: config changed from index -1 to 45, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:29.216665 6149 raft_consensus.cc:1261] T 18a448dbe2af493abefbc728a8f85186 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 85. Preceding OpId from leader: term: 1 index: 86. (index mismatch)
I20240502 07:58:29.217552 6770 consensus_queue.cc:1035] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 86, Last known committed idx: 84, Time since last communication: 0.000s
I20240502 07:58:29.221545 5756 raft_consensus.cc:1261] T 18a448dbe2af493abefbc728a8f85186 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 85. Preceding OpId from leader: term: 1 index: 86. (index mismatch)
I20240502 07:58:29.224260 7334 consensus_queue.cc:1035] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 86, Last known committed idx: 84, Time since last communication: 0.000s
I20240502 07:58:29.225271 7334 consensus_queue.cc:1035] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 86, Last known committed idx: 84, Time since last communication: 0.000s
I20240502 07:58:29.227241 6770 raft_consensus.cc:2932] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [term 1 LEADER]: Committing config change with OpId 1.86: config changed from index -1 to 86, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 86 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:29.229589 5483 catalog_manager.cc:5484] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 reported cstate change: config changed from index -1 to 86, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New cstate: current_term: 1 leader_uuid: "91e82003a656442189a329b29846a841" committed_config { opid_index: 86 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:29.236642 5756 raft_consensus.cc:2932] T 18a448dbe2af493abefbc728a8f85186 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Committing config change with OpId 1.86: config changed from index -1 to 86, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 86 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:29.237627 6021 raft_consensus.cc:2932] T 18a448dbe2af493abefbc728a8f85186 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Committing config change with OpId 1.86: config changed from index -1 to 86, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 86 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:29.242377 6152 raft_consensus.cc:2932] T 18a448dbe2af493abefbc728a8f85186 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.86: config changed from index -1 to 86, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 86 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:29.251376 7548 ts_tablet_manager.cc:902] T 7001a52697f1417bb84a0e3ed673ed0c P 27e3fafad7a24bcda362d2197be87017: Initiating tablet copy from peer d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70:41047)
I20240502 07:58:29.251263 5892 raft_consensus.cc:2932] T 18a448dbe2af493abefbc728a8f85186 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Committing config change with OpId 1.86: config changed from index -1 to 86, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 86 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:29.251968 7548 tablet_copy_client.cc:308] T 7001a52697f1417bb84a0e3ed673ed0c P 27e3fafad7a24bcda362d2197be87017: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.70:41047
I20240502 07:58:29.252429 6303 tablet_copy_service.cc:140] P d58f6d1ff8cf472f8a9edf3700e64bfb: Received BeginTabletCopySession request for tablet 7001a52697f1417bb84a0e3ed673ed0c from peer 27e3fafad7a24bcda362d2197be87017 ({username='slave'} at 127.3.139.71:33001)
I20240502 07:58:29.252524 6303 tablet_copy_service.cc:161] P d58f6d1ff8cf472f8a9edf3700e64bfb: Beginning new tablet copy session on tablet 7001a52697f1417bb84a0e3ed673ed0c from peer 27e3fafad7a24bcda362d2197be87017 at {username='slave'} at 127.3.139.71:33001: session id = 27e3fafad7a24bcda362d2197be87017-7001a52697f1417bb84a0e3ed673ed0c
I20240502 07:58:29.253458 6303 tablet_copy_source_session.cc:205] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:29.254503 7548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7001a52697f1417bb84a0e3ed673ed0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:29.260676 7548 tablet_copy_client.cc:783] T 7001a52697f1417bb84a0e3ed673ed0c P 27e3fafad7a24bcda362d2197be87017: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:29.261032 7548 tablet_copy_client.cc:647] T 7001a52697f1417bb84a0e3ed673ed0c P 27e3fafad7a24bcda362d2197be87017: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:29.303181 7548 tablet_copy_client.cc:523] T 7001a52697f1417bb84a0e3ed673ed0c P 27e3fafad7a24bcda362d2197be87017: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:29.304641 7548 tablet_bootstrap.cc:492] T 7001a52697f1417bb84a0e3ed673ed0c P 27e3fafad7a24bcda362d2197be87017: Bootstrap starting.
I20240502 07:58:29.338829 6020 consensus_queue.cc:237] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } }
I20240502 07:58:29.346572 6148 raft_consensus.cc:1261] T cd070443095547d1bf82a6c4a7666b86 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 07:58:29.347535 5958 consensus_peers.cc:485] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d -> Peer 27e3fafad7a24bcda362d2197be87017 (127.3.139.71:35341): Couldn't send request to peer 27e3fafad7a24bcda362d2197be87017. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cd070443095547d1bf82a6c4a7666b86. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:29.348332 5627 raft_consensus.cc:1261] T cd070443095547d1bf82a6c4a7666b86 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:29.350517 7556 consensus_queue.cc:1035] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:58:29.352062 6285 raft_consensus.cc:1261] T cd070443095547d1bf82a6c4a7666b86 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:29.352528 6515 consensus_queue.cc:1035] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:58:29.355409 5756 raft_consensus.cc:1261] T cd070443095547d1bf82a6c4a7666b86 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:29.358731 7556 consensus_queue.cc:1035] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:58:29.371729 7556 raft_consensus.cc:2932] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:29.373670 6515 consensus_queue.cc:1035] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:58:29.374689 6148 raft_consensus.cc:2932] T cd070443095547d1bf82a6c4a7666b86 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:29.377935 5483 catalog_manager.cc:5484] T cd070443095547d1bf82a6c4a7666b86 P e020e7ffe8ce423a8bdefec667161217 reported cstate change: config changed from index -1 to 2, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New cstate: current_term: 1 leader_uuid: "d5bc7863cd294f77a932c4a40f98b79d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:29.380954 6283 raft_consensus.cc:2932] T cd070443095547d1bf82a6c4a7666b86 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:29.391918 5628 raft_consensus.cc:2932] T cd070443095547d1bf82a6c4a7666b86 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:29.400732 5759 raft_consensus.cc:2932] T cd070443095547d1bf82a6c4a7666b86 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:29.593057 7548 tablet_bootstrap.cc:492] T 7001a52697f1417bb84a0e3ed673ed0c P 27e3fafad7a24bcda362d2197be87017: Bootstrap replayed 1/1 log segments. Stats: ops{read=88 overwritten=0 applied=87 ignored=0} inserts{seen=759 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240502 07:58:29.593475 7548 tablet_bootstrap.cc:492] T 7001a52697f1417bb84a0e3ed673ed0c P 27e3fafad7a24bcda362d2197be87017: Bootstrap complete.
I20240502 07:58:29.593883 7548 ts_tablet_manager.cc:1367] T 7001a52697f1417bb84a0e3ed673ed0c P 27e3fafad7a24bcda362d2197be87017: Time spent bootstrapping tablet: real 0.289s user 0.084s sys 0.000s
I20240502 07:58:29.594381 7548 raft_consensus.cc:348] T 7001a52697f1417bb84a0e3ed673ed0c P 27e3fafad7a24bcda362d2197be87017 [term 1 LEARNER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: 82 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } }
I20240502 07:58:29.594794 7548 raft_consensus.cc:729] T 7001a52697f1417bb84a0e3ed673ed0c P 27e3fafad7a24bcda362d2197be87017 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 27e3fafad7a24bcda362d2197be87017, State: Initialized, Role: LEARNER
I20240502 07:58:29.594955 7548 consensus_queue.cc:260] T 7001a52697f1417bb84a0e3ed673ed0c P 27e3fafad7a24bcda362d2197be87017 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 87, Last appended: 1.88, Last appended by leader: 88, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 82 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } }
I20240502 07:58:29.595649 7548 ts_tablet_manager.cc:1398] T 7001a52697f1417bb84a0e3ed673ed0c P 27e3fafad7a24bcda362d2197be87017: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 07:58:29.596459 6303 tablet_copy_service.cc:342] P d58f6d1ff8cf472f8a9edf3700e64bfb: Request end of tablet copy session 27e3fafad7a24bcda362d2197be87017-7001a52697f1417bb84a0e3ed673ed0c received from {username='slave'} at 127.3.139.71:33001
I20240502 07:58:29.596531 6303 tablet_copy_service.cc:433] P d58f6d1ff8cf472f8a9edf3700e64bfb: ending tablet copy session 27e3fafad7a24bcda362d2197be87017-7001a52697f1417bb84a0e3ed673ed0c on tablet 7001a52697f1417bb84a0e3ed673ed0c with peer 27e3fafad7a24bcda362d2197be87017
I20240502 07:58:29.685845 7548 ts_tablet_manager.cc:902] T 7f3d5d24bc164883adb1a11552500ce6 P 27e3fafad7a24bcda362d2197be87017: Initiating tablet copy from peer d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70:41047)
I20240502 07:58:29.686277 7548 tablet_copy_client.cc:308] T 7f3d5d24bc164883adb1a11552500ce6 P 27e3fafad7a24bcda362d2197be87017: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.70:41047
I20240502 07:58:29.686748 6303 tablet_copy_service.cc:140] P d58f6d1ff8cf472f8a9edf3700e64bfb: Received BeginTabletCopySession request for tablet 7f3d5d24bc164883adb1a11552500ce6 from peer 27e3fafad7a24bcda362d2197be87017 ({username='slave'} at 127.3.139.71:33001)
I20240502 07:58:29.686833 6303 tablet_copy_service.cc:161] P d58f6d1ff8cf472f8a9edf3700e64bfb: Beginning new tablet copy session on tablet 7f3d5d24bc164883adb1a11552500ce6 from peer 27e3fafad7a24bcda362d2197be87017 at {username='slave'} at 127.3.139.71:33001: session id = 27e3fafad7a24bcda362d2197be87017-7f3d5d24bc164883adb1a11552500ce6
I20240502 07:58:29.687813 6303 tablet_copy_source_session.cc:205] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:29.690685 7548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f3d5d24bc164883adb1a11552500ce6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:29.692905 7548 tablet_copy_client.cc:783] T 7f3d5d24bc164883adb1a11552500ce6 P 27e3fafad7a24bcda362d2197be87017: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:29.693132 7548 tablet_copy_client.cc:647] T 7f3d5d24bc164883adb1a11552500ce6 P 27e3fafad7a24bcda362d2197be87017: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:29.713791 7548 tablet_copy_client.cc:523] T 7f3d5d24bc164883adb1a11552500ce6 P 27e3fafad7a24bcda362d2197be87017: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:29.714957 7548 tablet_bootstrap.cc:492] T 7f3d5d24bc164883adb1a11552500ce6 P 27e3fafad7a24bcda362d2197be87017: Bootstrap starting.
I20240502 07:58:29.744953 7047 raft_consensus.cc:1203] T 7001a52697f1417bb84a0e3ed673ed0c P 27e3fafad7a24bcda362d2197be87017 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.81->[1.82-1.98] Dedup: 1.88->[1.89-1.98]
I20240502 07:58:29.868468 7548 tablet_bootstrap.cc:492] T 7f3d5d24bc164883adb1a11552500ce6 P 27e3fafad7a24bcda362d2197be87017: Bootstrap replayed 1/1 log segments. Stats: ops{read=48 overwritten=0 applied=48 ignored=0} inserts{seen=679 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:58:29.869065 7548 tablet_bootstrap.cc:492] T 7f3d5d24bc164883adb1a11552500ce6 P 27e3fafad7a24bcda362d2197be87017: Bootstrap complete.
I20240502 07:58:29.869428 7548 ts_tablet_manager.cc:1367] T 7f3d5d24bc164883adb1a11552500ce6 P 27e3fafad7a24bcda362d2197be87017: Time spent bootstrapping tablet: real 0.155s user 0.047s sys 0.007s
I20240502 07:58:29.869947 7548 raft_consensus.cc:348] T 7f3d5d24bc164883adb1a11552500ce6 P 27e3fafad7a24bcda362d2197be87017 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } }
I20240502 07:58:29.870165 7548 raft_consensus.cc:729] T 7f3d5d24bc164883adb1a11552500ce6 P 27e3fafad7a24bcda362d2197be87017 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 27e3fafad7a24bcda362d2197be87017, State: Initialized, Role: LEARNER
I20240502 07:58:29.870321 7548 consensus_queue.cc:260] T 7f3d5d24bc164883adb1a11552500ce6 P 27e3fafad7a24bcda362d2197be87017 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 48, Last appended: 1.48, Last appended by leader: 48, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } }
I20240502 07:58:29.871140 7548 ts_tablet_manager.cc:1398] T 7f3d5d24bc164883adb1a11552500ce6 P 27e3fafad7a24bcda362d2197be87017: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 07:58:29.871538 6303 tablet_copy_service.cc:342] P d58f6d1ff8cf472f8a9edf3700e64bfb: Request end of tablet copy session 27e3fafad7a24bcda362d2197be87017-7f3d5d24bc164883adb1a11552500ce6 received from {username='slave'} at 127.3.139.71:33001
I20240502 07:58:29.871623 6303 tablet_copy_service.cc:433] P d58f6d1ff8cf472f8a9edf3700e64bfb: ending tablet copy session 27e3fafad7a24bcda362d2197be87017-7f3d5d24bc164883adb1a11552500ce6 on tablet 7f3d5d24bc164883adb1a11552500ce6 with peer 27e3fafad7a24bcda362d2197be87017
I20240502 07:58:29.875281 7548 ts_tablet_manager.cc:902] T 18a448dbe2af493abefbc728a8f85186 P 27e3fafad7a24bcda362d2197be87017: Initiating tablet copy from peer 91e82003a656442189a329b29846a841 (127.3.139.65:43429)
I20240502 07:58:29.875703 7548 tablet_copy_client.cc:308] T 18a448dbe2af493abefbc728a8f85186 P 27e3fafad7a24bcda362d2197be87017: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.65:43429
I20240502 07:58:29.876701 5650 tablet_copy_service.cc:140] P 91e82003a656442189a329b29846a841: Received BeginTabletCopySession request for tablet 18a448dbe2af493abefbc728a8f85186 from peer 27e3fafad7a24bcda362d2197be87017 ({username='slave'} at 127.3.139.71:34345)
I20240502 07:58:29.876803 5650 tablet_copy_service.cc:161] P 91e82003a656442189a329b29846a841: Beginning new tablet copy session on tablet 18a448dbe2af493abefbc728a8f85186 from peer 27e3fafad7a24bcda362d2197be87017 at {username='slave'} at 127.3.139.71:34345: session id = 27e3fafad7a24bcda362d2197be87017-18a448dbe2af493abefbc728a8f85186
I20240502 07:58:29.877739 5650 tablet_copy_source_session.cc:205] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:29.878845 7548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18a448dbe2af493abefbc728a8f85186. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:29.881408 7548 tablet_copy_client.cc:783] T 18a448dbe2af493abefbc728a8f85186 P 27e3fafad7a24bcda362d2197be87017: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:29.881587 7548 tablet_copy_client.cc:647] T 18a448dbe2af493abefbc728a8f85186 P 27e3fafad7a24bcda362d2197be87017: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:29.977739 7613 ts_tablet_manager.cc:902] T cd070443095547d1bf82a6c4a7666b86 P 27e3fafad7a24bcda362d2197be87017: Initiating tablet copy from peer d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68:44287)
I20240502 07:58:29.978433 7613 tablet_copy_client.cc:308] T cd070443095547d1bf82a6c4a7666b86 P 27e3fafad7a24bcda362d2197be87017: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.68:44287
I20240502 07:58:29.978816 6043 tablet_copy_service.cc:140] P d5bc7863cd294f77a932c4a40f98b79d: Received BeginTabletCopySession request for tablet cd070443095547d1bf82a6c4a7666b86 from peer 27e3fafad7a24bcda362d2197be87017 ({username='slave'} at 127.3.139.71:34815)
I20240502 07:58:29.978902 6043 tablet_copy_service.cc:161] P d5bc7863cd294f77a932c4a40f98b79d: Beginning new tablet copy session on tablet cd070443095547d1bf82a6c4a7666b86 from peer 27e3fafad7a24bcda362d2197be87017 at {username='slave'} at 127.3.139.71:34815: session id = 27e3fafad7a24bcda362d2197be87017-cd070443095547d1bf82a6c4a7666b86
I20240502 07:58:29.979797 6043 tablet_copy_source_session.cc:205] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:29.989238 6285 consensus_queue.cc:237] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 78, Committed index: 78, Last appended: 1.78, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 79 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } }
I20240502 07:58:29.991299 5890 raft_consensus.cc:1261] T 07d53332bdd34233b3653756770dc9cf P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 78. Preceding OpId from leader: term: 1 index: 79. (index mismatch)
I20240502 07:58:29.991410 5761 raft_consensus.cc:1261] T 07d53332bdd34233b3653756770dc9cf P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 78. Preceding OpId from leader: term: 1 index: 79. (index mismatch)
I20240502 07:58:29.991609 5626 raft_consensus.cc:1261] T 07d53332bdd34233b3653756770dc9cf P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 78. Preceding OpId from leader: term: 1 index: 79. (index mismatch)
I20240502 07:58:29.991662 6674 consensus_queue.cc:1035] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 79, Last known committed idx: 78, Time since last communication: 0.000s
I20240502 07:58:29.992457 7308 consensus_queue.cc:1035] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 79, Last known committed idx: 78, Time since last communication: 0.000s
I20240502 07:58:29.992699 7308 consensus_queue.cc:1035] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 79, Last known committed idx: 78, Time since last communication: 0.000s
I20240502 07:58:29.994576 7308 raft_consensus.cc:1050] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb: attempting to promote NON_VOTER 27e3fafad7a24bcda362d2197be87017 to VOTER
I20240502 07:58:29.994846 6153 raft_consensus.cc:1261] T 07d53332bdd34233b3653756770dc9cf P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 78. Preceding OpId from leader: term: 1 index: 79. (index mismatch)
I20240502 07:58:29.994901 7308 consensus_queue.cc:237] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 104, Committed index: 104, Last appended: 1.104, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 105 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } }
I20240502 07:58:29.996950 5626 raft_consensus.cc:1261] T 7001a52697f1417bb84a0e3ed673ed0c P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 104. Preceding OpId from leader: term: 1 index: 105. (index mismatch)
I20240502 07:58:29.996994 6928 raft_consensus.cc:2932] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 LEADER]: Committing config change with OpId 1.79: config changed from index -1 to 79, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 79 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:29.999675 7613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd070443095547d1bf82a6c4a7666b86. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:30.000140 5482 catalog_manager.cc:5484] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb reported cstate change: config changed from index -1 to 79, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New cstate: current_term: 1 leader_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" committed_config { opid_index: 79 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:30.001961 5889 raft_consensus.cc:2932] T 07d53332bdd34233b3653756770dc9cf P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Committing config change with OpId 1.79: config changed from index -1 to 79, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 79 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:30.004359 7308 consensus_queue.cc:1035] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 79, Last known committed idx: 78, Time since last communication: 0.000s
I20240502 07:58:30.004720 6674 consensus_queue.cc:1035] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 105, Last known committed idx: 104, Time since last communication: 0.000s
I20240502 07:58:30.005509 6150 raft_consensus.cc:2932] T 07d53332bdd34233b3653756770dc9cf P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.79: config changed from index -1 to 79, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 79 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:30.006004 5890 raft_consensus.cc:1261] T 7001a52697f1417bb84a0e3ed673ed0c P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 104. Preceding OpId from leader: term: 1 index: 105. (index mismatch)
I20240502 07:58:30.008487 5628 raft_consensus.cc:2932] T 07d53332bdd34233b3653756770dc9cf P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Committing config change with OpId 1.79: config changed from index -1 to 79, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 79 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:30.012609 7548 tablet_copy_client.cc:523] T 18a448dbe2af493abefbc728a8f85186 P 27e3fafad7a24bcda362d2197be87017: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:30.013886 7548 tablet_bootstrap.cc:492] T 18a448dbe2af493abefbc728a8f85186 P 27e3fafad7a24bcda362d2197be87017: Bootstrap starting.
I20240502 07:58:30.017824 5756 raft_consensus.cc:2932] T 07d53332bdd34233b3653756770dc9cf P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Committing config change with OpId 1.79: config changed from index -1 to 79, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 79 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:30.019521 7253 consensus_queue.cc:1035] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 105, Last known committed idx: 104, Time since last communication: 0.000s
I20240502 07:58:30.029014 7613 tablet_copy_client.cc:783] T cd070443095547d1bf82a6c4a7666b86 P 27e3fafad7a24bcda362d2197be87017: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:30.029163 7613 tablet_copy_client.cc:647] T cd070443095547d1bf82a6c4a7666b86 P 27e3fafad7a24bcda362d2197be87017: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:30.030541 7045 raft_consensus.cc:1261] T 7001a52697f1417bb84a0e3ed673ed0c P 27e3fafad7a24bcda362d2197be87017 [term 1 LEARNER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 104. Preceding OpId from leader: term: 1 index: 105. (index mismatch)
W20240502 07:58:30.031780 6220 consensus_peers.cc:485] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb -> Peer 27e3fafad7a24bcda362d2197be87017 (127.3.139.71:35341): Couldn't send request to peer 27e3fafad7a24bcda362d2197be87017. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 07d53332bdd34233b3653756770dc9cf. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:30.031975 7610 consensus_queue.cc:1035] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 105, Last known committed idx: 104, Time since last communication: 0.000s
I20240502 07:58:30.033063 7613 tablet_copy_client.cc:523] T cd070443095547d1bf82a6c4a7666b86 P 27e3fafad7a24bcda362d2197be87017: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:30.034219 7613 tablet_bootstrap.cc:492] T cd070443095547d1bf82a6c4a7666b86 P 27e3fafad7a24bcda362d2197be87017: Bootstrap starting.
I20240502 07:58:30.039132 7253 raft_consensus.cc:2932] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 LEADER]: Committing config change with OpId 1.105: config changed from index 82 to 105, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 105 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:30.042197 7045 raft_consensus.cc:2932] T 7001a52697f1417bb84a0e3ed673ed0c P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Committing config change with OpId 1.105: config changed from index 82 to 105, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 105 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:30.045658 5486 catalog_manager.cc:5484] T 7001a52697f1417bb84a0e3ed673ed0c P 27e3fafad7a24bcda362d2197be87017 reported cstate change: config changed from index 82 to 105, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" committed_config { opid_index: 105 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:30.045665 5892 raft_consensus.cc:2932] T 7001a52697f1417bb84a0e3ed673ed0c P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Committing config change with OpId 1.105: config changed from index 82 to 105, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 105 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:30.049926 7613 tablet_bootstrap.cc:492] T cd070443095547d1bf82a6c4a7666b86 P 27e3fafad7a24bcda362d2197be87017: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:58:30.050205 7613 tablet_bootstrap.cc:492] T cd070443095547d1bf82a6c4a7666b86 P 27e3fafad7a24bcda362d2197be87017: Bootstrap complete.
I20240502 07:58:30.050374 7613 ts_tablet_manager.cc:1367] T cd070443095547d1bf82a6c4a7666b86 P 27e3fafad7a24bcda362d2197be87017: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.000s
I20240502 07:58:30.050806 7613 raft_consensus.cc:348] T cd070443095547d1bf82a6c4a7666b86 P 27e3fafad7a24bcda362d2197be87017 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } }
I20240502 07:58:30.050992 7613 raft_consensus.cc:729] T cd070443095547d1bf82a6c4a7666b86 P 27e3fafad7a24bcda362d2197be87017 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 27e3fafad7a24bcda362d2197be87017, State: Initialized, Role: LEARNER
I20240502 07:58:30.051187 7613 consensus_queue.cc:260] T cd070443095547d1bf82a6c4a7666b86 P 27e3fafad7a24bcda362d2197be87017 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } }
I20240502 07:58:30.051533 7613 ts_tablet_manager.cc:1398] T cd070443095547d1bf82a6c4a7666b86 P 27e3fafad7a24bcda362d2197be87017: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:30.051921 6043 tablet_copy_service.cc:342] P d5bc7863cd294f77a932c4a40f98b79d: Request end of tablet copy session 27e3fafad7a24bcda362d2197be87017-cd070443095547d1bf82a6c4a7666b86 received from {username='slave'} at 127.3.139.71:34815
I20240502 07:58:30.052009 6043 tablet_copy_service.cc:433] P d5bc7863cd294f77a932c4a40f98b79d: ending tablet copy session 27e3fafad7a24bcda362d2197be87017-cd070443095547d1bf82a6c4a7666b86 on tablet cd070443095547d1bf82a6c4a7666b86 with peer 27e3fafad7a24bcda362d2197be87017
I20240502 07:58:30.057487 6153 raft_consensus.cc:1261] T 7001a52697f1417bb84a0e3ed673ed0c P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 104. Preceding OpId from leader: term: 1 index: 105. (index mismatch)
I20240502 07:58:30.057958 7610 consensus_queue.cc:1035] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 105, Last known committed idx: 104, Time since last communication: 0.000s
I20240502 07:58:30.066072 5626 raft_consensus.cc:2932] T 7001a52697f1417bb84a0e3ed673ed0c P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Committing config change with OpId 1.105: config changed from index 82 to 105, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 105 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:30.069298 7045 raft_consensus.cc:1203] T 7f3d5d24bc164883adb1a11552500ce6 P 27e3fafad7a24bcda362d2197be87017 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.44->[1.45-1.52] Dedup: 1.48->[1.49-1.52]
I20240502 07:58:30.076346 6151 raft_consensus.cc:2932] T 7001a52697f1417bb84a0e3ed673ed0c P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.105: config changed from index 82 to 105, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 105 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:30.096397 6022 raft_consensus.cc:1261] T 7001a52697f1417bb84a0e3ed673ed0c P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 101. Preceding OpId from leader: term: 1 index: 105. (index mismatch)
I20240502 07:58:30.097030 7610 consensus_queue.cc:1035] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 105, Last known committed idx: 100, Time since last communication: 0.000s
I20240502 07:58:30.098537 6022 raft_consensus.cc:2932] T 7001a52697f1417bb84a0e3ed673ed0c P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Committing config change with OpId 1.105: config changed from index 82 to 105, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 105 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:30.113566 6285 consensus_queue.cc:237] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 106, Committed index: 106, Last appended: 1.107, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 108 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } }
I20240502 07:58:30.114812 7044 raft_consensus.cc:1261] T 7001a52697f1417bb84a0e3ed673ed0c P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 107. Preceding OpId from leader: term: 1 index: 108. (index mismatch)
I20240502 07:58:30.115612 6018 raft_consensus.cc:1261] T 7001a52697f1417bb84a0e3ed673ed0c P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 107. Preceding OpId from leader: term: 1 index: 108. (index mismatch)
I20240502 07:58:30.115785 7610 consensus_queue.cc:1035] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 108, Last known committed idx: 106, Time since last communication: 0.000s
I20240502 07:58:30.116686 7610 consensus_queue.cc:1035] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 108, Last known committed idx: 106, Time since last communication: 0.000s
I20240502 07:58:30.117005 5889 raft_consensus.cc:1261] T 7001a52697f1417bb84a0e3ed673ed0c P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 107. Preceding OpId from leader: term: 1 index: 108. (index mismatch)
I20240502 07:58:30.117326 7610 consensus_queue.cc:1035] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 108, Last known committed idx: 106, Time since last communication: 0.000s
I20240502 07:58:30.118499 7610 raft_consensus.cc:2932] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 LEADER]: Committing config change with OpId 1.108: config changed from index 105 to 108, VOTER e020e7ffe8ce423a8bdefec667161217 (127.3.139.69) evicted. New config: { opid_index: 108 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:30.121526 5484 catalog_manager.cc:5484] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb reported cstate change: config changed from index 105 to 108, VOTER e020e7ffe8ce423a8bdefec667161217 (127.3.139.69) evicted. New cstate: current_term: 1 leader_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" committed_config { opid_index: 108 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:58:30.122201 5474 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 7001a52697f1417bb84a0e3ed673ed0c with cas_config_opid_index 105: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:58:30.122347 7048 raft_consensus.cc:2932] T 7001a52697f1417bb84a0e3ed673ed0c P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Committing config change with OpId 1.108: config changed from index 105 to 108, VOTER e020e7ffe8ce423a8bdefec667161217 (127.3.139.69) evicted. New config: { opid_index: 108 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:30.123864 5889 raft_consensus.cc:2932] T 7001a52697f1417bb84a0e3ed673ed0c P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Committing config change with OpId 1.108: config changed from index 105 to 108, VOTER e020e7ffe8ce423a8bdefec667161217 (127.3.139.69) evicted. New config: { opid_index: 108 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:30.140774 6132 tablet_service.cc:1514] Processing DeleteTablet for tablet 7001a52697f1417bb84a0e3ed673ed0c with delete_type TABLET_DATA_TOMBSTONED (TS e020e7ffe8ce423a8bdefec667161217 not found in new config with opid_index 108) from {username='slave'} at 127.0.0.1:46442
I20240502 07:58:30.154606 6020 raft_consensus.cc:2932] T 7001a52697f1417bb84a0e3ed673ed0c P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Committing config change with OpId 1.108: config changed from index 105 to 108, VOTER e020e7ffe8ce423a8bdefec667161217 (127.3.139.69) evicted. New config: { opid_index: 108 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:30.168815 7548 tablet_bootstrap.cc:492] T 18a448dbe2af493abefbc728a8f85186 P 27e3fafad7a24bcda362d2197be87017: Bootstrap replayed 1/1 log segments. Stats: ops{read=96 overwritten=0 applied=95 ignored=0} inserts{seen=780 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240502 07:58:30.169076 7548 tablet_bootstrap.cc:492] T 18a448dbe2af493abefbc728a8f85186 P 27e3fafad7a24bcda362d2197be87017: Bootstrap complete.
I20240502 07:58:30.169248 7548 ts_tablet_manager.cc:1367] T 18a448dbe2af493abefbc728a8f85186 P 27e3fafad7a24bcda362d2197be87017: Time spent bootstrapping tablet: real 0.155s user 0.081s sys 0.005s
I20240502 07:58:30.169690 7548 raft_consensus.cc:348] T 18a448dbe2af493abefbc728a8f85186 P 27e3fafad7a24bcda362d2197be87017 [term 1 LEARNER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: 86 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } }
I20240502 07:58:30.169965 7548 raft_consensus.cc:729] T 18a448dbe2af493abefbc728a8f85186 P 27e3fafad7a24bcda362d2197be87017 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 27e3fafad7a24bcda362d2197be87017, State: Initialized, Role: LEARNER
I20240502 07:58:30.170078 7548 consensus_queue.cc:260] T 18a448dbe2af493abefbc728a8f85186 P 27e3fafad7a24bcda362d2197be87017 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 95, Last appended: 1.96, Last appended by leader: 96, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 86 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } }
I20240502 07:58:30.170665 7548 ts_tablet_manager.cc:1398] T 18a448dbe2af493abefbc728a8f85186 P 27e3fafad7a24bcda362d2197be87017: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240502 07:58:30.193588 5628 raft_consensus.cc:1261] T 7001a52697f1417bb84a0e3ed673ed0c P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 105. Preceding OpId from leader: term: 1 index: 108. (index mismatch)
I20240502 07:58:30.195168 7610 raft_consensus.cc:1050] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb: attempting to promote NON_VOTER 27e3fafad7a24bcda362d2197be87017 to VOTER
I20240502 07:58:30.195523 7610 consensus_queue.cc:237] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 55, Committed index: 55, Last appended: 1.56, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 57 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } }
I20240502 07:58:30.197185 5756 raft_consensus.cc:1261] T 7f3d5d24bc164883adb1a11552500ce6 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 56. Preceding OpId from leader: term: 1 index: 57. (index mismatch)
I20240502 07:58:30.197599 5892 raft_consensus.cc:1261] T 7f3d5d24bc164883adb1a11552500ce6 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 55. Preceding OpId from leader: term: 1 index: 57. (index mismatch)
I20240502 07:58:30.197582 6674 consensus_queue.cc:1035] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 57, Last known committed idx: 55, Time since last communication: 0.000s
I20240502 07:58:30.200181 7044 raft_consensus.cc:1261] T 7f3d5d24bc164883adb1a11552500ce6 P 27e3fafad7a24bcda362d2197be87017 [term 1 LEARNER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 55. Preceding OpId from leader: term: 1 index: 57. (index mismatch)
I20240502 07:58:30.201289 6928 consensus_queue.cc:1035] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 57, Last known committed idx: 54, Time since last communication: 0.000s
I20240502 07:58:30.201675 6674 consensus_queue.cc:1035] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 57, Last known committed idx: 54, Time since last communication: 0.000s
I20240502 07:58:30.202587 7045 raft_consensus.cc:1203] T 7f3d5d24bc164883adb1a11552500ce6 P 27e3fafad7a24bcda362d2197be87017 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.55->[1.56-1.57] Dedup: 1.56->[1.57-1.57]
I20240502 07:58:30.206652 6674 raft_consensus.cc:2932] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 LEADER]: Committing config change with OpId 1.57: config changed from index 45 to 57, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 57 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:30.207330 6020 raft_consensus.cc:1261] T 7f3d5d24bc164883adb1a11552500ce6 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 55. Preceding OpId from leader: term: 1 index: 57. (index mismatch)
I20240502 07:58:30.207983 6674 consensus_queue.cc:1035] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 57, Last known committed idx: 54, Time since last communication: 0.000s
I20240502 07:58:30.208222 5892 raft_consensus.cc:2932] T 7f3d5d24bc164883adb1a11552500ce6 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Committing config change with OpId 1.57: config changed from index 45 to 57, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 57 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:30.208688 6674 consensus_queue.cc:1035] T 7001a52697f1417bb84a0e3ed673ed0c P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 108, Last known committed idx: 105, Time since last communication: 0.000s
I20240502 07:58:30.210990 5488 catalog_manager.cc:5484] T 7f3d5d24bc164883adb1a11552500ce6 P 89bf5c0a6b2a4d949851089446f17b96 reported cstate change: config changed from index 45 to 57, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" committed_config { opid_index: 57 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:30.211581 7045 raft_consensus.cc:2932] T 7f3d5d24bc164883adb1a11552500ce6 P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Committing config change with OpId 1.57: config changed from index 45 to 57, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 57 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:30.212745 5761 raft_consensus.cc:2932] T 7f3d5d24bc164883adb1a11552500ce6 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Committing config change with OpId 1.57: config changed from index 45 to 57, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 57 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:30.218153 6020 raft_consensus.cc:2932] T 7f3d5d24bc164883adb1a11552500ce6 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Committing config change with OpId 1.57: config changed from index 45 to 57, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 57 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:30.234249 5650 tablet_copy_service.cc:342] P 91e82003a656442189a329b29846a841: Request end of tablet copy session 27e3fafad7a24bcda362d2197be87017-18a448dbe2af493abefbc728a8f85186 received from {username='slave'} at 127.3.139.71:34345
I20240502 07:58:30.234329 5650 tablet_copy_service.cc:433] P 91e82003a656442189a329b29846a841: ending tablet copy session 27e3fafad7a24bcda362d2197be87017-18a448dbe2af493abefbc728a8f85186 on tablet 18a448dbe2af493abefbc728a8f85186 with peer 27e3fafad7a24bcda362d2197be87017
I20240502 07:58:30.261467 5626 raft_consensus.cc:1261] T 7f3d5d24bc164883adb1a11552500ce6 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 56. Preceding OpId from leader: term: 1 index: 57. (index mismatch)
I20240502 07:58:30.271374 5629 raft_consensus.cc:1203] T 7001a52697f1417bb84a0e3ed673ed0c P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.105->[1.106-1.112] Dedup: 1.106->[1.107-1.112]
I20240502 07:58:30.273151 5629 raft_consensus.cc:2932] T 7001a52697f1417bb84a0e3ed673ed0c P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Committing config change with OpId 1.108: config changed from index 105 to 108, VOTER e020e7ffe8ce423a8bdefec667161217 (127.3.139.69) evicted. New config: { opid_index: 108 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:30.307302 7650 tablet_replica.cc:330] T 7001a52697f1417bb84a0e3ed673ed0c P e020e7ffe8ce423a8bdefec667161217: stopping tablet replica
I20240502 07:58:30.307592 7650 raft_consensus.cc:2229] T 7001a52697f1417bb84a0e3ed673ed0c P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:30.307722 7650 pending_rounds.cc:62] T 7001a52697f1417bb84a0e3ed673ed0c P e020e7ffe8ce423a8bdefec667161217: Trying to abort 1 pending ops.
I20240502 07:58:30.307760 7650 pending_rounds.cc:68] T 7001a52697f1417bb84a0e3ed673ed0c P e020e7ffe8ce423a8bdefec667161217: Aborting op as it isn't in flight: id { term: 1 index: 107 } timestamp: 7023151964602617856 op_type: WRITE_OP write_request { tablet_id: "7001a52697f1417bb84a0e3ed673ed0c" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\007\000^,\326(\351.\240s\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000" indirect_data: "<redacted>""fd343f2162395dd6" } external_consistency_mode: CLIENT_PROPAGATED propagated_timestamp: 7023151964577263616 authz_token { token_data: "\010\313\221\315\261\006\"3\n\005slave\022*\n 6f5d44f6fbe544099b33ad94e35cee6e\020\001\030\001 \001(\001" signature: "<redacted>""w\242UF\371,\371\340\013\017\266\270\032i\254\374+9\235\'JV\013\334\205\031;(\005|\330|\240\353(nm\260\362\336\221\251n\254\2773u\301K\350\006\375\243\356\322*FW\324\272G\350\204\273" signing_key_seq_num: 0 } } request_id { client_id: "85d95aa7aaf24d5d9156d56c8068ca7a" seq_no: 335 first_incomplete_seq_no: 335 attempt_no: 0 }
I20240502 07:58:30.308053 7650 raft_consensus.cc:2258] T 7001a52697f1417bb84a0e3ed673ed0c P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:30.314424 7650 ts_tablet_manager.cc:1875] T 7001a52697f1417bb84a0e3ed673ed0c P e020e7ffe8ce423a8bdefec667161217: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:58:30.322185 7650 ts_tablet_manager.cc:1888] T 7001a52697f1417bb84a0e3ed673ed0c P e020e7ffe8ce423a8bdefec667161217: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.107
I20240502 07:58:30.322269 7650 log.cc:1196] T 7001a52697f1417bb84a0e3ed673ed0c P e020e7ffe8ce423a8bdefec667161217: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/wal/wals/7001a52697f1417bb84a0e3ed673ed0c
I20240502 07:58:30.323957 5474 catalog_manager.cc:4830] TS e020e7ffe8ce423a8bdefec667161217 (127.3.139.69:39399): tablet 7001a52697f1417bb84a0e3ed673ed0c (table rebalance_test_table_1 [id=6f5d44f6fbe544099b33ad94e35cee6e]) successfully deleted
I20240502 07:58:30.327641 7610 consensus_queue.cc:1035] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 57, Last known committed idx: 55, Time since last communication: 0.000s
I20240502 07:58:30.333108 7044 raft_consensus.cc:1203] T 18a448dbe2af493abefbc728a8f85186 P 27e3fafad7a24bcda362d2197be87017 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.85->[1.86-1.101] Dedup: 1.96->[1.97-1.101]
I20240502 07:58:30.339181 5628 raft_consensus.cc:2932] T 7f3d5d24bc164883adb1a11552500ce6 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Committing config change with OpId 1.57: config changed from index 45 to 57, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 57 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:30.422641 7047 raft_consensus.cc:1203] T cd070443095547d1bf82a6c4a7666b86 P 27e3fafad7a24bcda362d2197be87017 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 07:58:30.466002 6740 raft_consensus.cc:1050] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841: attempting to promote NON_VOTER 27e3fafad7a24bcda362d2197be87017 to VOTER
I20240502 07:58:30.474036 6285 consensus_queue.cc:237] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 58, Committed index: 58, Last appended: 1.58, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } }
I20240502 07:58:30.475502 5891 raft_consensus.cc:1261] T 7f3d5d24bc164883adb1a11552500ce6 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 58. Preceding OpId from leader: term: 1 index: 59. (index mismatch)
I20240502 07:58:30.475739 6022 raft_consensus.cc:1261] T 7f3d5d24bc164883adb1a11552500ce6 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 58. Preceding OpId from leader: term: 1 index: 59. (index mismatch)
I20240502 07:58:30.476012 7610 consensus_queue.cc:1035] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 59, Last known committed idx: 58, Time since last communication: 0.000s
I20240502 07:58:30.476608 7048 raft_consensus.cc:1261] T 7f3d5d24bc164883adb1a11552500ce6 P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 58. Preceding OpId from leader: term: 1 index: 59. (index mismatch)
I20240502 07:58:30.477006 5761 raft_consensus.cc:1261] T 7f3d5d24bc164883adb1a11552500ce6 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 58. Preceding OpId from leader: term: 1 index: 59. (index mismatch)
I20240502 07:58:30.477622 7610 consensus_queue.cc:1035] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 59, Last known committed idx: 57, Time since last communication: 0.000s
I20240502 07:58:30.477726 7253 consensus_queue.cc:1035] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 59, Last known committed idx: 58, Time since last communication: 0.000s
I20240502 07:58:30.478238 7253 consensus_queue.cc:1035] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 59, Last known committed idx: 58, Time since last communication: 0.000s
I20240502 07:58:30.484835 6740 consensus_queue.cc:237] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 101, Committed index: 101, Last appended: 1.102, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 103 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } }
I20240502 07:58:30.486294 6928 raft_consensus.cc:2932] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 LEADER]: Committing config change with OpId 1.59: config changed from index 57 to 59, VOTER 91e82003a656442189a329b29846a841 (127.3.139.65) evicted. New config: { opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:30.488744 6022 raft_consensus.cc:1261] T 18a448dbe2af493abefbc728a8f85186 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 102. Preceding OpId from leader: term: 1 index: 103. (index mismatch)
I20240502 07:58:30.490203 5474 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 7f3d5d24bc164883adb1a11552500ce6 with cas_config_opid_index 57: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:58:30.490681 7048 raft_consensus.cc:1261] T 18a448dbe2af493abefbc728a8f85186 P 27e3fafad7a24bcda362d2197be87017 [term 1 LEARNER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 102. Preceding OpId from leader: term: 1 index: 103. (index mismatch)
I20240502 07:58:30.490694 5891 raft_consensus.cc:1261] T 18a448dbe2af493abefbc728a8f85186 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 102. Preceding OpId from leader: term: 1 index: 103. (index mismatch)
I20240502 07:58:30.491308 6740 consensus_queue.cc:1035] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 103, Last known committed idx: 102, Time since last communication: 0.000s
I20240502 07:58:30.491452 7334 consensus_queue.cc:1035] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 103, Last known committed idx: 101, Time since last communication: 0.000s
I20240502 07:58:30.491776 6740 consensus_queue.cc:1035] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 103, Last known committed idx: 101, Time since last communication: 0.000s
I20240502 07:58:30.493347 5488 catalog_manager.cc:5484] T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb reported cstate change: config changed from index 57 to 59, VOTER 91e82003a656442189a329b29846a841 (127.3.139.65) evicted. New cstate: current_term: 1 leader_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" committed_config { opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:30.493404 7044 raft_consensus.cc:2932] T 7f3d5d24bc164883adb1a11552500ce6 P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Committing config change with OpId 1.59: config changed from index 57 to 59, VOTER 91e82003a656442189a329b29846a841 (127.3.139.65) evicted. New config: { opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:30.495497 6022 consensus_queue.cc:237] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 99, Committed index: 99, Last appended: 1.99, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 100 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } }
I20240502 07:58:30.496232 5608 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f3d5d24bc164883adb1a11552500ce6 with delete_type TABLET_DATA_TOMBSTONED (TS 91e82003a656442189a329b29846a841 not found in new config with opid_index 59) from {username='slave'} at 127.0.0.1:54448
I20240502 07:58:30.498505 5890 raft_consensus.cc:1261] T d14545ba1e474960909a1b140b6a315c P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 99. Preceding OpId from leader: term: 1 index: 100. (index mismatch)
I20240502 07:58:30.498742 5759 raft_consensus.cc:1261] T d14545ba1e474960909a1b140b6a315c P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 99. Preceding OpId from leader: term: 1 index: 100. (index mismatch)
W20240502 07:58:30.498872 5958 consensus_peers.cc:485] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d -> Peer 27e3fafad7a24bcda362d2197be87017 (127.3.139.71:35341): Couldn't send request to peer 27e3fafad7a24bcda362d2197be87017. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d14545ba1e474960909a1b140b6a315c. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:30.498983 7657 consensus_queue.cc:1035] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 100, Last known committed idx: 99, Time since last communication: 0.000s
I20240502 07:58:30.500130 5761 raft_consensus.cc:1261] T 18a448dbe2af493abefbc728a8f85186 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 102. Preceding OpId from leader: term: 1 index: 103. (index mismatch)
I20240502 07:58:30.500236 7657 consensus_queue.cc:1035] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 100, Last known committed idx: 99, Time since last communication: 0.000s
I20240502 07:58:30.502041 5626 raft_consensus.cc:1261] T d14545ba1e474960909a1b140b6a315c P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 99. Preceding OpId from leader: term: 1 index: 100. (index mismatch)
I20240502 07:58:30.502596 7556 consensus_queue.cc:1035] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 100, Last known committed idx: 99, Time since last communication: 0.000s
I20240502 07:58:30.503479 7676 consensus_queue.cc:1035] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 103, Last known committed idx: 102, Time since last communication: 0.000s
I20240502 07:58:30.504525 7157 raft_consensus.cc:2932] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [term 1 LEADER]: Committing config change with OpId 1.100: config changed from index -1 to 100, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 100 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:30.507920 5488 catalog_manager.cc:5484] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d reported cstate change: config changed from index -1 to 100, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New cstate: current_term: 1 leader_uuid: "d5bc7863cd294f77a932c4a40f98b79d" committed_config { opid_index: 100 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:30.509742 6150 raft_consensus.cc:1261] T 18a448dbe2af493abefbc728a8f85186 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 101. Preceding OpId from leader: term: 1 index: 103. (index mismatch)
I20240502 07:58:30.513052 7677 consensus_queue.cc:1035] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 103, Last known committed idx: 101, Time since last communication: 0.000s
I20240502 07:58:30.516515 5891 raft_consensus.cc:2932] T d14545ba1e474960909a1b140b6a315c P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Committing config change with OpId 1.100: config changed from index -1 to 100, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 100 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:30.517171 6150 raft_consensus.cc:1203] T 18a448dbe2af493abefbc728a8f85186 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.101->[1.102-1.104] Dedup: 1.102->[1.103-1.104]
I20240502 07:58:30.522701 6152 raft_consensus.cc:1261] T d14545ba1e474960909a1b140b6a315c P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 99. Preceding OpId from leader: term: 1 index: 100. (index mismatch)
I20240502 07:58:30.523842 7678 tablet_replica.cc:330] T 7f3d5d24bc164883adb1a11552500ce6 P 91e82003a656442189a329b29846a841: stopping tablet replica
I20240502 07:58:30.524156 7678 raft_consensus.cc:2229] T 7f3d5d24bc164883adb1a11552500ce6 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:30.524410 7678 raft_consensus.cc:2258] T 7f3d5d24bc164883adb1a11552500ce6 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:30.527140 7274 consensus_queue.cc:1035] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 100, Last known committed idx: 98, Time since last communication: 0.000s
I20240502 07:58:30.527310 6740 raft_consensus.cc:2932] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [term 1 LEADER]: Committing config change with OpId 1.103: config changed from index 86 to 103, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 103 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:30.527667 5626 raft_consensus.cc:2932] T d14545ba1e474960909a1b140b6a315c P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Committing config change with OpId 1.100: config changed from index -1 to 100, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 100 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:30.528424 6152 raft_consensus.cc:2932] T d14545ba1e474960909a1b140b6a315c P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.100: config changed from index -1 to 100, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 100 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:30.530016 6150 raft_consensus.cc:2932] T 18a448dbe2af493abefbc728a8f85186 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.103: config changed from index 86 to 103, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 103 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:30.535321 5761 raft_consensus.cc:2932] T 18a448dbe2af493abefbc728a8f85186 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Committing config change with OpId 1.103: config changed from index 86 to 103, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 103 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:30.537240 5757 raft_consensus.cc:2932] T d14545ba1e474960909a1b140b6a315c P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Committing config change with OpId 1.100: config changed from index -1 to 100, NON_VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) added. New config: { opid_index: 100 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } }
I20240502 07:58:30.541833 5892 raft_consensus.cc:2932] T 7f3d5d24bc164883adb1a11552500ce6 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Committing config change with OpId 1.59: config changed from index 57 to 59, VOTER 91e82003a656442189a329b29846a841 (127.3.139.65) evicted. New config: { opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:30.543507 6020 raft_consensus.cc:2932] T 7f3d5d24bc164883adb1a11552500ce6 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Committing config change with OpId 1.59: config changed from index 57 to 59, VOTER 91e82003a656442189a329b29846a841 (127.3.139.65) evicted. New config: { opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:30.543335 5756 raft_consensus.cc:2932] T 7f3d5d24bc164883adb1a11552500ce6 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Committing config change with OpId 1.59: config changed from index 57 to 59, VOTER 91e82003a656442189a329b29846a841 (127.3.139.65) evicted. New config: { opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:30.553962 5480 catalog_manager.cc:5484] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 reported cstate change: config changed from index 86 to 103, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "91e82003a656442189a329b29846a841" committed_config { opid_index: 103 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:30.556205 6018 raft_consensus.cc:2932] T 18a448dbe2af493abefbc728a8f85186 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Committing config change with OpId 1.103: config changed from index 86 to 103, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 103 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:30.556666 5890 raft_consensus.cc:2932] T 18a448dbe2af493abefbc728a8f85186 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Committing config change with OpId 1.103: config changed from index 86 to 103, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 103 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:30.584565 7678 ts_tablet_manager.cc:1875] T 7f3d5d24bc164883adb1a11552500ce6 P 91e82003a656442189a329b29846a841: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:58:30.596776 5626 consensus_queue.cc:237] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 104, Committed index: 104, Last appended: 1.105, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 106 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } }
I20240502 07:58:30.597525 7044 raft_consensus.cc:2932] T 18a448dbe2af493abefbc728a8f85186 P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Committing config change with OpId 1.103: config changed from index 86 to 103, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 103 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:30.598508 6018 raft_consensus.cc:1261] T 18a448dbe2af493abefbc728a8f85186 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 105. Preceding OpId from leader: term: 1 index: 106. (index mismatch)
I20240502 07:58:30.598865 7482 consensus_queue.cc:1035] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 106, Last known committed idx: 104, Time since last communication: 0.000s
I20240502 07:58:30.610545 7678 ts_tablet_manager.cc:1888] T 7f3d5d24bc164883adb1a11552500ce6 P 91e82003a656442189a329b29846a841: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.58
I20240502 07:58:30.610623 7678 log.cc:1196] T 7f3d5d24bc164883adb1a11552500ce6 P 91e82003a656442189a329b29846a841: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal/wals/7f3d5d24bc164883adb1a11552500ce6
I20240502 07:58:30.611366 5473 catalog_manager.cc:4830] TS 91e82003a656442189a329b29846a841 (127.3.139.65:43429): tablet 7f3d5d24bc164883adb1a11552500ce6 (table rebalance_test_table_4 [id=5c1e49d6edf446d5a06063e568da2a0a]) successfully deleted
I20240502 07:58:30.625156 7548 ts_tablet_manager.cc:902] T 07d53332bdd34233b3653756770dc9cf P 27e3fafad7a24bcda362d2197be87017: Initiating tablet copy from peer d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70:41047)
I20240502 07:58:30.625603 7548 tablet_copy_client.cc:308] T 07d53332bdd34233b3653756770dc9cf P 27e3fafad7a24bcda362d2197be87017: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.70:41047
I20240502 07:58:30.626116 6303 tablet_copy_service.cc:140] P d58f6d1ff8cf472f8a9edf3700e64bfb: Received BeginTabletCopySession request for tablet 07d53332bdd34233b3653756770dc9cf from peer 27e3fafad7a24bcda362d2197be87017 ({username='slave'} at 127.3.139.71:33001)
I20240502 07:58:30.626219 6303 tablet_copy_service.cc:161] P d58f6d1ff8cf472f8a9edf3700e64bfb: Beginning new tablet copy session on tablet 07d53332bdd34233b3653756770dc9cf from peer 27e3fafad7a24bcda362d2197be87017 at {username='slave'} at 127.3.139.71:33001: session id = 27e3fafad7a24bcda362d2197be87017-07d53332bdd34233b3653756770dc9cf
I20240502 07:58:30.627118 6303 tablet_copy_source_session.cc:205] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:30.628000 7548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07d53332bdd34233b3653756770dc9cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:30.636638 7548 tablet_copy_client.cc:783] T 07d53332bdd34233b3653756770dc9cf P 27e3fafad7a24bcda362d2197be87017: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:30.636976 7548 tablet_copy_client.cc:647] T 07d53332bdd34233b3653756770dc9cf P 27e3fafad7a24bcda362d2197be87017: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:30.639644 7048 raft_consensus.cc:1261] T 18a448dbe2af493abefbc728a8f85186 P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 105. Preceding OpId from leader: term: 1 index: 106. (index mismatch)
I20240502 07:58:30.640214 6740 consensus_queue.cc:1035] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 106, Last known committed idx: 104, Time since last communication: 0.000s
I20240502 07:58:30.642526 6151 raft_consensus.cc:1261] T 18a448dbe2af493abefbc728a8f85186 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 105. Preceding OpId from leader: term: 1 index: 106. (index mismatch)
I20240502 07:58:30.643575 7325 raft_consensus.cc:2932] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [term 1 LEADER]: Committing config change with OpId 1.106: config changed from index 103 to 106, VOTER eda55a8566c447cfa749457825b70688 (127.3.139.66) evicted. New config: { opid_index: 106 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:30.645288 7482 consensus_queue.cc:1035] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 106, Last known committed idx: 104, Time since last communication: 0.000s
I20240502 07:58:30.646595 7045 raft_consensus.cc:2932] T 18a448dbe2af493abefbc728a8f85186 P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Committing config change with OpId 1.106: config changed from index 103 to 106, VOTER eda55a8566c447cfa749457825b70688 (127.3.139.66) evicted. New config: { opid_index: 106 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:30.654799 6151 raft_consensus.cc:2932] T 18a448dbe2af493abefbc728a8f85186 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.106: config changed from index 103 to 106, VOTER eda55a8566c447cfa749457825b70688 (127.3.139.66) evicted. New config: { opid_index: 106 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:30.703281 5473 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 18a448dbe2af493abefbc728a8f85186 with cas_config_opid_index 103: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:58:30.707267 6020 raft_consensus.cc:2932] T 18a448dbe2af493abefbc728a8f85186 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Committing config change with OpId 1.106: config changed from index 103 to 106, VOTER eda55a8566c447cfa749457825b70688 (127.3.139.66) evicted. New config: { opid_index: 106 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:30.707340 5488 catalog_manager.cc:5484] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 reported cstate change: config changed from index 103 to 106, VOTER eda55a8566c447cfa749457825b70688 (127.3.139.66) evicted. New cstate: current_term: 1 leader_uuid: "91e82003a656442189a329b29846a841" committed_config { opid_index: 106 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:58:30.739799 7548 tablet_copy_client.cc:523] T 07d53332bdd34233b3653756770dc9cf P 27e3fafad7a24bcda362d2197be87017: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20240502 07:58:30.798620 6959 log.cc:924] Time spent T 7f3d5d24bc164883adb1a11552500ce6 P 89bf5c0a6b2a4d949851089446f17b96: Append to log took a long time: real 0.065s user 0.000s sys 0.000s
W20240502 07:58:30.798646 6954 log.cc:924] Time spent T 7f3d5d24bc164883adb1a11552500ce6 P d5bc7863cd294f77a932c4a40f98b79d: Append to log took a long time: real 0.064s user 0.000s sys 0.000s
W20240502 07:58:30.798759 6958 log.cc:924] Time spent T 7f3d5d24bc164883adb1a11552500ce6 P eda55a8566c447cfa749457825b70688: Append to log took a long time: real 0.065s user 0.000s sys 0.000s
W20240502 07:58:30.798852 6933 log.cc:924] Time spent T 07d53332bdd34233b3653756770dc9cf P e020e7ffe8ce423a8bdefec667161217: Append to log took a long time: real 0.082s user 0.000s sys 0.000s
W20240502 07:58:30.799018 6888 log.cc:924] Time spent T 7f3d5d24bc164883adb1a11552500ce6 P d58f6d1ff8cf472f8a9edf3700e64bfb: Append to log took a long time: real 0.080s user 0.000s sys 0.000s
I20240502 07:58:30.800658 7548 tablet_bootstrap.cc:492] T 07d53332bdd34233b3653756770dc9cf P 27e3fafad7a24bcda362d2197be87017: Bootstrap starting.
I20240502 07:58:30.805352 5740 tablet_service.cc:1514] Processing DeleteTablet for tablet 18a448dbe2af493abefbc728a8f85186 with delete_type TABLET_DATA_TOMBSTONED (TS eda55a8566c447cfa749457825b70688 not found in new config with opid_index 106) from {username='slave'} at 127.0.0.1:41882
I20240502 07:58:30.805783 5888 raft_consensus.cc:1261] T 18a448dbe2af493abefbc728a8f85186 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 104. Preceding OpId from leader: term: 1 index: 106. (index mismatch)
I20240502 07:58:30.857331 7676 consensus_queue.cc:1035] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 106, Last known committed idx: 104, Time since last communication: 0.000s
I20240502 07:58:30.906328 7556 raft_consensus.cc:1050] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d: attempting to promote NON_VOTER 27e3fafad7a24bcda362d2197be87017 to VOTER
I20240502 07:58:30.906764 7556 consensus_queue.cc:237] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } }
I20240502 07:58:30.911127 6284 raft_consensus.cc:1261] T cd070443095547d1bf82a6c4a7666b86 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 07:58:30.911471 7657 consensus_queue.cc:1035] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:58:30.915609 6151 raft_consensus.cc:1261] T cd070443095547d1bf82a6c4a7666b86 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 07:58:30.916038 7657 consensus_queue.cc:1035] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:58:30.922106 7048 raft_consensus.cc:1261] T cd070443095547d1bf82a6c4a7666b86 P 27e3fafad7a24bcda362d2197be87017 [term 1 LEARNER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 07:58:30.927728 7658 consensus_queue.cc:1035] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:58:30.936522 5761 raft_consensus.cc:1261] T cd070443095547d1bf82a6c4a7666b86 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 07:58:30.938233 7274 consensus_queue.cc:1035] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:58:30.968286 5626 raft_consensus.cc:1261] T cd070443095547d1bf82a6c4a7666b86 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 07:58:30.971931 5888 raft_consensus.cc:2932] T 18a448dbe2af493abefbc728a8f85186 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Committing config change with OpId 1.106: config changed from index 103 to 106, VOTER eda55a8566c447cfa749457825b70688 (127.3.139.66) evicted. New config: { opid_index: 106 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:30.980342 7274 consensus_queue.cc:1035] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:58:31.022330 7710 tablet_replica.cc:330] T 18a448dbe2af493abefbc728a8f85186 P eda55a8566c447cfa749457825b70688: stopping tablet replica
I20240502 07:58:31.022480 7710 raft_consensus.cc:2229] T 18a448dbe2af493abefbc728a8f85186 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:31.022724 7710 pending_rounds.cc:62] T 18a448dbe2af493abefbc728a8f85186 P eda55a8566c447cfa749457825b70688: Trying to abort 1 pending ops.
I20240502 07:58:31.022763 7710 pending_rounds.cc:68] T 18a448dbe2af493abefbc728a8f85186 P eda55a8566c447cfa749457825b70688: Aborting op as it isn't in flight: id { term: 1 index: 105 } timestamp: 7023151966501249024 op_type: WRITE_OP write_request { tablet_id: "18a448dbe2af493abefbc728a8f85186" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\007\000\021\210`B\367\032\2452\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000" indirect_data: "<redacted>""8b56dfa0e8e75008" } external_consistency_mode: CLIENT_PROPAGATED propagated_timestamp: 7023151966166257664 authz_token { token_data: "\010\313\221\315\261\006\"3\n\005slave\022*\n 28039a31ea894d23af67c1f1da387f21\020\001\030\001 \001(\001" signature: "<redacted>""{\270\001\374g\317\235\353\373\261\005 P\263C\305V\260\216w|\014\226\006n\363\204\262N\272\035t\231V\241\246\266\327\252\242W\245\362\335\264\321\344f\320!V3\027\333\264\005VCu$J7\031\307" signing_key_seq_num: 0 } } request_id { client_id: "287602d09b79464390e8dc120d5fba10" seq_no: 291 first_incomplete_seq_no: 291 attempt_no: 0 }
I20240502 07:58:31.023243 7717 ts_tablet_manager.cc:902] T d14545ba1e474960909a1b140b6a315c P 27e3fafad7a24bcda362d2197be87017: Initiating tablet copy from peer d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68:44287)
I20240502 07:58:31.023772 7717 tablet_copy_client.cc:308] T d14545ba1e474960909a1b140b6a315c P 27e3fafad7a24bcda362d2197be87017: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.68:44287
I20240502 07:58:31.024156 6043 tablet_copy_service.cc:140] P d5bc7863cd294f77a932c4a40f98b79d: Received BeginTabletCopySession request for tablet d14545ba1e474960909a1b140b6a315c from peer 27e3fafad7a24bcda362d2197be87017 ({username='slave'} at 127.3.139.71:34815)
I20240502 07:58:31.024243 6043 tablet_copy_service.cc:161] P d5bc7863cd294f77a932c4a40f98b79d: Beginning new tablet copy session on tablet d14545ba1e474960909a1b140b6a315c from peer 27e3fafad7a24bcda362d2197be87017 at {username='slave'} at 127.3.139.71:34815: session id = 27e3fafad7a24bcda362d2197be87017-d14545ba1e474960909a1b140b6a315c
I20240502 07:58:31.025110 6043 tablet_copy_source_session.cc:205] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:31.027878 7717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d14545ba1e474960909a1b140b6a315c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:31.023013 7710 raft_consensus.cc:2258] T 18a448dbe2af493abefbc728a8f85186 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Raft consensus is shut down!
W20240502 07:58:31.097978 5564 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.139.66:39925, user_credentials={real_user=slave}} blocked reactor thread for 56118.9us
I20240502 07:58:31.099673 7556 raft_consensus.cc:2932] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:31.109288 5480 catalog_manager.cc:5484] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d reported cstate change: config changed from index 2 to 3, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d5bc7863cd294f77a932c4a40f98b79d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:58:31.110091 7710 ts_tablet_manager.cc:1875] T 18a448dbe2af493abefbc728a8f85186 P eda55a8566c447cfa749457825b70688: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:58:31.115865 5629 raft_consensus.cc:2932] T cd070443095547d1bf82a6c4a7666b86 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:31.146171 6281 raft_consensus.cc:2932] T cd070443095547d1bf82a6c4a7666b86 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
W20240502 07:58:31.188146 7713 log.cc:924] Time spent T cd070443095547d1bf82a6c4a7666b86 P e020e7ffe8ce423a8bdefec667161217: Append to log took a long time: real 0.075s user 0.000s sys 0.000s
I20240502 07:58:31.194765 6150 raft_consensus.cc:2932] T cd070443095547d1bf82a6c4a7666b86 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:31.111017 7047 raft_consensus.cc:2932] T cd070443095547d1bf82a6c4a7666b86 P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:31.200476 5761 raft_consensus.cc:2932] T cd070443095547d1bf82a6c4a7666b86 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:31.203151 6018 consensus_queue.cc:237] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } }
I20240502 07:58:31.212863 7045 raft_consensus.cc:1261] T cd070443095547d1bf82a6c4a7666b86 P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:58:31.213290 7657 consensus_queue.cc:1035] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:58:31.030504 7717 tablet_copy_client.cc:783] T d14545ba1e474960909a1b140b6a315c P 27e3fafad7a24bcda362d2197be87017: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:31.213541 7717 tablet_copy_client.cc:647] T d14545ba1e474960909a1b140b6a315c P 27e3fafad7a24bcda362d2197be87017: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:31.217101 7717 tablet_copy_client.cc:523] T d14545ba1e474960909a1b140b6a315c P 27e3fafad7a24bcda362d2197be87017: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:31.217818 6149 raft_consensus.cc:1261] T cd070443095547d1bf82a6c4a7666b86 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:58:31.218711 7657 consensus_queue.cc:1035] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:58:31.225629 5757 raft_consensus.cc:1261] T cd070443095547d1bf82a6c4a7666b86 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:58:31.226186 7548 tablet_bootstrap.cc:492] T 07d53332bdd34233b3653756770dc9cf P 27e3fafad7a24bcda362d2197be87017: Bootstrap replayed 1/1 log segments. Stats: ops{read=84 overwritten=0 applied=84 ignored=0} inserts{seen=756 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:58:31.226482 7548 tablet_bootstrap.cc:492] T 07d53332bdd34233b3653756770dc9cf P 27e3fafad7a24bcda362d2197be87017: Bootstrap complete.
I20240502 07:58:31.226719 7548 ts_tablet_manager.cc:1367] T 07d53332bdd34233b3653756770dc9cf P 27e3fafad7a24bcda362d2197be87017: Time spent bootstrapping tablet: real 0.426s user 0.069s sys 0.010s
I20240502 07:58:31.227005 5629 raft_consensus.cc:1261] T cd070443095547d1bf82a6c4a7666b86 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:58:31.227330 7548 raft_consensus.cc:348] T 07d53332bdd34233b3653756770dc9cf P 27e3fafad7a24bcda362d2197be87017 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 79 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } }
I20240502 07:58:31.227516 7548 raft_consensus.cc:729] T 07d53332bdd34233b3653756770dc9cf P 27e3fafad7a24bcda362d2197be87017 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 27e3fafad7a24bcda362d2197be87017, State: Initialized, Role: LEARNER
I20240502 07:58:31.227700 7548 consensus_queue.cc:260] T 07d53332bdd34233b3653756770dc9cf P 27e3fafad7a24bcda362d2197be87017 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 84, Last appended: 1.84, Last appended by leader: 84, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 79 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } }
I20240502 07:58:31.228777 7548 ts_tablet_manager.cc:1398] T 07d53332bdd34233b3653756770dc9cf P 27e3fafad7a24bcda362d2197be87017: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:31.229207 6303 tablet_copy_service.cc:342] P d58f6d1ff8cf472f8a9edf3700e64bfb: Request end of tablet copy session 27e3fafad7a24bcda362d2197be87017-07d53332bdd34233b3653756770dc9cf received from {username='slave'} at 127.3.139.71:33001
I20240502 07:58:31.229275 6303 tablet_copy_service.cc:433] P d58f6d1ff8cf472f8a9edf3700e64bfb: ending tablet copy session 27e3fafad7a24bcda362d2197be87017-07d53332bdd34233b3653756770dc9cf on tablet 07d53332bdd34233b3653756770dc9cf with peer 27e3fafad7a24bcda362d2197be87017
I20240502 07:58:31.230552 7658 consensus_queue.cc:1035] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:58:31.236724 7556 consensus_queue.cc:1035] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:58:31.236969 7556 raft_consensus.cc:2932] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:31.245702 5757 raft_consensus.cc:2932] T cd070443095547d1bf82a6c4a7666b86 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:31.250133 5626 raft_consensus.cc:2932] T cd070443095547d1bf82a6c4a7666b86 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:31.255179 6149 raft_consensus.cc:2932] T cd070443095547d1bf82a6c4a7666b86 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:31.302865 7717 tablet_bootstrap.cc:492] T d14545ba1e474960909a1b140b6a315c P 27e3fafad7a24bcda362d2197be87017: Bootstrap starting.
I20240502 07:58:31.304530 7710 ts_tablet_manager.cc:1888] T 18a448dbe2af493abefbc728a8f85186 P eda55a8566c447cfa749457825b70688: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.105
I20240502 07:58:31.304612 7710 log.cc:1196] T 18a448dbe2af493abefbc728a8f85186 P eda55a8566c447cfa749457825b70688: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal/wals/18a448dbe2af493abefbc728a8f85186
I20240502 07:58:31.305732 5473 catalog_manager.cc:4830] TS eda55a8566c447cfa749457825b70688 (127.3.139.66:39925): tablet 18a448dbe2af493abefbc728a8f85186 (table rebalance_test_table_2 [id=28039a31ea894d23af67c1f1da387f21]) successfully deleted
I20240502 07:58:31.309909 5488 catalog_manager.cc:5484] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d reported cstate change: config changed from index 3 to 4, VOTER d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70) evicted. New cstate: current_term: 1 leader_uuid: "d5bc7863cd294f77a932c4a40f98b79d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:58:31.310528 5473 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet cd070443095547d1bf82a6c4a7666b86 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:58:31.310930 7045 raft_consensus.cc:2932] T cd070443095547d1bf82a6c4a7666b86 P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:31.313459 6263 tablet_service.cc:1514] Processing DeleteTablet for tablet cd070443095547d1bf82a6c4a7666b86 with delete_type TABLET_DATA_TOMBSTONED (TS d58f6d1ff8cf472f8a9edf3700e64bfb not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33086
I20240502 07:58:31.368865 7729 tablet_replica.cc:330] T cd070443095547d1bf82a6c4a7666b86 P d58f6d1ff8cf472f8a9edf3700e64bfb: stopping tablet replica
I20240502 07:58:31.431288 7729 raft_consensus.cc:2229] T cd070443095547d1bf82a6c4a7666b86 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:31.431560 7729 raft_consensus.cc:2258] T cd070443095547d1bf82a6c4a7666b86 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:31.458287 7729 ts_tablet_manager.cc:1875] T cd070443095547d1bf82a6c4a7666b86 P d58f6d1ff8cf472f8a9edf3700e64bfb: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:58:31.460527 7729 ts_tablet_manager.cc:1888] T cd070443095547d1bf82a6c4a7666b86 P d58f6d1ff8cf472f8a9edf3700e64bfb: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 07:58:31.460597 7729 log.cc:1196] T cd070443095547d1bf82a6c4a7666b86 P d58f6d1ff8cf472f8a9edf3700e64bfb: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/wal/wals/cd070443095547d1bf82a6c4a7666b86
I20240502 07:58:31.477921 5474 catalog_manager.cc:4830] TS d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70:41047): tablet cd070443095547d1bf82a6c4a7666b86 (table TestTable [id=ce12803e6d704c758d74d2e1fb8a39b4]) successfully deleted
I20240502 07:58:31.610653 7717 tablet_bootstrap.cc:492] T d14545ba1e474960909a1b140b6a315c P 27e3fafad7a24bcda362d2197be87017: Bootstrap replayed 1/1 log segments. Stats: ops{read=113 overwritten=0 applied=113 ignored=0} inserts{seen=798 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:58:31.611039 7717 tablet_bootstrap.cc:492] T d14545ba1e474960909a1b140b6a315c P 27e3fafad7a24bcda362d2197be87017: Bootstrap complete.
I20240502 07:58:31.611301 7717 ts_tablet_manager.cc:1367] T d14545ba1e474960909a1b140b6a315c P 27e3fafad7a24bcda362d2197be87017: Time spent bootstrapping tablet: real 0.308s user 0.098s sys 0.005s
I20240502 07:58:31.613004 7717 raft_consensus.cc:348] T d14545ba1e474960909a1b140b6a315c P 27e3fafad7a24bcda362d2197be87017 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 100 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } }
I20240502 07:58:31.613360 7717 raft_consensus.cc:729] T d14545ba1e474960909a1b140b6a315c P 27e3fafad7a24bcda362d2197be87017 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 27e3fafad7a24bcda362d2197be87017, State: Initialized, Role: LEARNER
I20240502 07:58:31.613512 7717 consensus_queue.cc:260] T d14545ba1e474960909a1b140b6a315c P 27e3fafad7a24bcda362d2197be87017 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 113, Last appended: 1.113, Last appended by leader: 113, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 100 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } }
I20240502 07:58:31.616475 7717 ts_tablet_manager.cc:1398] T d14545ba1e474960909a1b140b6a315c P 27e3fafad7a24bcda362d2197be87017: Time spent starting tablet: real 0.005s user 0.000s sys 0.001s
I20240502 07:58:31.616922 6043 tablet_copy_service.cc:342] P d5bc7863cd294f77a932c4a40f98b79d: Request end of tablet copy session 27e3fafad7a24bcda362d2197be87017-d14545ba1e474960909a1b140b6a315c received from {username='slave'} at 127.3.139.71:34815
I20240502 07:58:31.616995 6043 tablet_copy_service.cc:433] P d5bc7863cd294f77a932c4a40f98b79d: ending tablet copy session 27e3fafad7a24bcda362d2197be87017-d14545ba1e474960909a1b140b6a315c on tablet d14545ba1e474960909a1b140b6a315c with peer 27e3fafad7a24bcda362d2197be87017
I20240502 07:58:31.748744 7045 raft_consensus.cc:1203] T 07d53332bdd34233b3653756770dc9cf P 27e3fafad7a24bcda362d2197be87017 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.78->[1.79-1.97] Dedup: 1.84->[1.85-1.97]
I20240502 07:58:31.835325 6674 raft_consensus.cc:1050] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb: attempting to promote NON_VOTER 27e3fafad7a24bcda362d2197be87017 to VOTER
I20240502 07:58:31.837112 6674 consensus_queue.cc:237] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 97, Committed index: 97, Last appended: 1.97, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 98 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } }
I20240502 07:58:31.839198 6149 raft_consensus.cc:1261] T 07d53332bdd34233b3653756770dc9cf P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 97. Preceding OpId from leader: term: 1 index: 98. (index mismatch)
I20240502 07:58:31.839807 7308 consensus_queue.cc:1035] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 98, Last known committed idx: 97, Time since last communication: 0.000s
I20240502 07:58:31.840134 7045 raft_consensus.cc:1261] T 07d53332bdd34233b3653756770dc9cf P 27e3fafad7a24bcda362d2197be87017 [term 1 LEARNER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 97. Preceding OpId from leader: term: 1 index: 98. (index mismatch)
I20240502 07:58:31.840502 7308 consensus_queue.cc:1035] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 98, Last known committed idx: 97, Time since last communication: 0.000s
I20240502 07:58:31.846254 5761 raft_consensus.cc:1261] T 07d53332bdd34233b3653756770dc9cf P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 97. Preceding OpId from leader: term: 1 index: 98. (index mismatch)
I20240502 07:58:31.847090 5892 raft_consensus.cc:1261] T 07d53332bdd34233b3653756770dc9cf P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 97. Preceding OpId from leader: term: 1 index: 98. (index mismatch)
I20240502 07:58:31.849763 7610 consensus_queue.cc:1035] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 98, Last known committed idx: 97, Time since last communication: 0.000s
I20240502 07:58:31.854882 7308 consensus_queue.cc:1035] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 98, Last known committed idx: 96, Time since last communication: 0.000s
I20240502 07:58:31.862293 7308 raft_consensus.cc:2932] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 LEADER]: Committing config change with OpId 1.98: config changed from index 79 to 98, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 98 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:31.865193 7045 raft_consensus.cc:2932] T 07d53332bdd34233b3653756770dc9cf P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Committing config change with OpId 1.98: config changed from index 79 to 98, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 98 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:31.865355 6151 raft_consensus.cc:2932] T 07d53332bdd34233b3653756770dc9cf P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.98: config changed from index 79 to 98, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 98 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:31.864154 5761 raft_consensus.cc:2932] T 07d53332bdd34233b3653756770dc9cf P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Committing config change with OpId 1.98: config changed from index 79 to 98, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 98 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:31.866449 5484 catalog_manager.cc:5484] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb reported cstate change: config changed from index 79 to 98, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" committed_config { opid_index: 98 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:58:31.878796 5892 raft_consensus.cc:2932] T 07d53332bdd34233b3653756770dc9cf P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Committing config change with OpId 1.98: config changed from index 79 to 98, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 98 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:31.884804 5629 raft_consensus.cc:1261] T 07d53332bdd34233b3653756770dc9cf P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 97. Preceding OpId from leader: term: 1 index: 98. (index mismatch)
I20240502 07:58:31.910569 7045 raft_consensus.cc:1203] T d14545ba1e474960909a1b140b6a315c P 27e3fafad7a24bcda362d2197be87017 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.99->[1.100-1.134] Dedup: 1.113->[1.114-1.134]
I20240502 07:58:31.933269 7308 consensus_queue.cc:1035] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 98, Last known committed idx: 96, Time since last communication: 0.000s
I20240502 07:58:31.956832 5626 raft_consensus.cc:2932] T 07d53332bdd34233b3653756770dc9cf P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Committing config change with OpId 1.98: config changed from index 79 to 98, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 98 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:31.980875 6283 consensus_queue.cc:237] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 101, Committed index: 101, Last appended: 1.101, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 102 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } }
I20240502 07:58:31.984587 7045 raft_consensus.cc:1261] T 07d53332bdd34233b3653756770dc9cf P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 101. Preceding OpId from leader: term: 1 index: 102. (index mismatch)
I20240502 07:58:31.986799 7308 consensus_queue.cc:1035] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 102, Last known committed idx: 101, Time since last communication: 0.000s
I20240502 07:58:31.990110 6149 raft_consensus.cc:1261] T 07d53332bdd34233b3653756770dc9cf P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 101. Preceding OpId from leader: term: 1 index: 102. (index mismatch)
I20240502 07:58:31.991467 5890 raft_consensus.cc:1261] T 07d53332bdd34233b3653756770dc9cf P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 101. Preceding OpId from leader: term: 1 index: 102. (index mismatch)
I20240502 07:58:31.991681 7219 consensus_queue.cc:1035] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 102, Last known committed idx: 101, Time since last communication: 0.000s
I20240502 07:58:31.992372 7219 consensus_queue.cc:1035] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 102, Last known committed idx: 101, Time since last communication: 0.000s
I20240502 07:58:31.997887 7253 raft_consensus.cc:2932] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 LEADER]: Committing config change with OpId 1.102: config changed from index 98 to 102, VOTER eda55a8566c447cfa749457825b70688 (127.3.139.66) evicted. New config: { opid_index: 102 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:32.000506 5486 catalog_manager.cc:5484] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb reported cstate change: config changed from index 98 to 102, VOTER eda55a8566c447cfa749457825b70688 (127.3.139.66) evicted. New cstate: current_term: 1 leader_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" committed_config { opid_index: 102 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:58:32.001813 5474 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 07d53332bdd34233b3653756770dc9cf with cas_config_opid_index 98: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:58:32.002482 6149 raft_consensus.cc:2932] T 07d53332bdd34233b3653756770dc9cf P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.102: config changed from index 98 to 102, VOTER eda55a8566c447cfa749457825b70688 (127.3.139.66) evicted. New config: { opid_index: 102 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:32.003526 5740 tablet_service.cc:1514] Processing DeleteTablet for tablet 07d53332bdd34233b3653756770dc9cf with delete_type TABLET_DATA_TOMBSTONED (TS eda55a8566c447cfa749457825b70688 not found in new config with opid_index 102) from {username='slave'} at 127.0.0.1:41882
I20240502 07:58:32.004820 7047 raft_consensus.cc:2932] T 07d53332bdd34233b3653756770dc9cf P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Committing config change with OpId 1.102: config changed from index 98 to 102, VOTER eda55a8566c447cfa749457825b70688 (127.3.139.66) evicted. New config: { opid_index: 102 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:32.014227 5628 raft_consensus.cc:1261] T 07d53332bdd34233b3653756770dc9cf P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 101. Preceding OpId from leader: term: 1 index: 102. (index mismatch)
I20240502 07:58:32.032128 5891 raft_consensus.cc:2932] T 07d53332bdd34233b3653756770dc9cf P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Committing config change with OpId 1.102: config changed from index 98 to 102, VOTER eda55a8566c447cfa749457825b70688 (127.3.139.66) evicted. New config: { opid_index: 102 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:32.032475 7157 raft_consensus.cc:1050] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d: attempting to promote NON_VOTER 27e3fafad7a24bcda362d2197be87017 to VOTER
I20240502 07:58:32.032806 7157 consensus_queue.cc:237] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 135, Committed index: 135, Last appended: 1.136, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 137 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } }
I20240502 07:58:32.036307 7047 raft_consensus.cc:1261] T d14545ba1e474960909a1b140b6a315c P 27e3fafad7a24bcda362d2197be87017 [term 1 LEARNER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 136. Preceding OpId from leader: term: 1 index: 137. (index mismatch)
I20240502 07:58:32.038165 6153 raft_consensus.cc:1261] T d14545ba1e474960909a1b140b6a315c P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 136. Preceding OpId from leader: term: 1 index: 137. (index mismatch)
I20240502 07:58:32.038429 7757 consensus_queue.cc:1035] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 137, Last known committed idx: 135, Time since last communication: 0.000s
I20240502 07:58:32.039264 5761 raft_consensus.cc:1261] T d14545ba1e474960909a1b140b6a315c P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 136. Preceding OpId from leader: term: 1 index: 137. (index mismatch)
I20240502 07:58:32.039848 5891 raft_consensus.cc:1261] T d14545ba1e474960909a1b140b6a315c P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 136. Preceding OpId from leader: term: 1 index: 137. (index mismatch)
I20240502 07:58:32.041617 7219 consensus_queue.cc:1035] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 102, Last known committed idx: 101, Time since last communication: 0.000s
I20240502 07:58:32.041905 5626 raft_consensus.cc:1261] T d14545ba1e474960909a1b140b6a315c P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 136. Preceding OpId from leader: term: 1 index: 137. (index mismatch)
I20240502 07:58:32.044162 7657 consensus_queue.cc:1035] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 137, Last known committed idx: 135, Time since last communication: 0.000s
I20240502 07:58:32.053745 7657 consensus_queue.cc:1035] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 137, Last known committed idx: 135, Time since last communication: 0.000s
I20240502 07:58:32.054698 5628 raft_consensus.cc:2932] T 07d53332bdd34233b3653756770dc9cf P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Committing config change with OpId 1.102: config changed from index 98 to 102, VOTER eda55a8566c447cfa749457825b70688 (127.3.139.66) evicted. New config: { opid_index: 102 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:32.063242 7657 consensus_queue.cc:1035] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 137, Last known committed idx: 135, Time since last communication: 0.000s
I20240502 07:58:32.063486 7657 consensus_queue.cc:1035] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 137, Last known committed idx: 135, Time since last communication: 0.000s
I20240502 07:58:32.069442 7774 tablet_replica.cc:330] T 07d53332bdd34233b3653756770dc9cf P eda55a8566c447cfa749457825b70688: stopping tablet replica
I20240502 07:58:32.073846 7157 raft_consensus.cc:2932] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [term 1 LEADER]: Committing config change with OpId 1.137: config changed from index 100 to 137, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 137 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:32.077137 5486 catalog_manager.cc:5484] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d reported cstate change: config changed from index 100 to 137, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d5bc7863cd294f77a932c4a40f98b79d" committed_config { opid_index: 137 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:58:32.078037 7774 raft_consensus.cc:2229] T 07d53332bdd34233b3653756770dc9cf P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:32.080437 5890 raft_consensus.cc:2932] T d14545ba1e474960909a1b140b6a315c P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Committing config change with OpId 1.137: config changed from index 100 to 137, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 137 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:32.083564 6018 consensus_queue.cc:237] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 137, Committed index: 137, Last appended: 1.137, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 138 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } }
I20240502 07:58:32.084365 7774 raft_consensus.cc:2258] T 07d53332bdd34233b3653756770dc9cf P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:32.085827 6153 raft_consensus.cc:2932] T d14545ba1e474960909a1b140b6a315c P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.137: config changed from index 100 to 137, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 137 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:32.087829 5628 raft_consensus.cc:2932] T d14545ba1e474960909a1b140b6a315c P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Committing config change with OpId 1.137: config changed from index 100 to 137, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 137 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:32.094026 7047 raft_consensus.cc:2932] T d14545ba1e474960909a1b140b6a315c P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Committing config change with OpId 1.137: config changed from index 100 to 137, 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 137 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:32.098281 5891 raft_consensus.cc:1261] T d14545ba1e474960909a1b140b6a315c P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 137. Preceding OpId from leader: term: 1 index: 138. (index mismatch)
I20240502 07:58:32.099373 5628 raft_consensus.cc:1261] T d14545ba1e474960909a1b140b6a315c P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 137. Preceding OpId from leader: term: 1 index: 138. (index mismatch)
I20240502 07:58:32.105455 6153 raft_consensus.cc:1261] T d14545ba1e474960909a1b140b6a315c P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 137. Preceding OpId from leader: term: 1 index: 138. (index mismatch)
I20240502 07:58:32.107693 7774 ts_tablet_manager.cc:1875] T 07d53332bdd34233b3653756770dc9cf P eda55a8566c447cfa749457825b70688: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:58:32.110741 7657 consensus_queue.cc:1035] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 138, Last known committed idx: 137, Time since last communication: 0.000s
I20240502 07:58:32.111543 7157 consensus_queue.cc:1035] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 138, Last known committed idx: 137, Time since last communication: 0.000s
I20240502 07:58:32.112349 7657 consensus_queue.cc:1035] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 138, Last known committed idx: 137, Time since last communication: 0.000s
I20240502 07:58:32.115583 7045 raft_consensus.cc:1261] T d14545ba1e474960909a1b140b6a315c P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 137. Preceding OpId from leader: term: 1 index: 138. (index mismatch)
I20240502 07:58:32.124716 7658 consensus_queue.cc:1035] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 138, Last known committed idx: 137, Time since last communication: 0.000s
I20240502 07:58:32.129058 7774 ts_tablet_manager.cc:1888] T 07d53332bdd34233b3653756770dc9cf P eda55a8566c447cfa749457825b70688: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.101
I20240502 07:58:32.129163 7774 log.cc:1196] T 07d53332bdd34233b3653756770dc9cf P eda55a8566c447cfa749457825b70688: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal/wals/07d53332bdd34233b3653756770dc9cf
I20240502 07:58:32.129720 5473 catalog_manager.cc:4830] TS eda55a8566c447cfa749457825b70688 (127.3.139.66:39925): tablet 07d53332bdd34233b3653756770dc9cf (table rebalance_test_table_4 [id=5c1e49d6edf446d5a06063e568da2a0a]) successfully deleted
I20240502 07:58:32.131597 7157 raft_consensus.cc:2932] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [term 1 LEADER]: Committing config change with OpId 1.138: config changed from index 137 to 138, VOTER eda55a8566c447cfa749457825b70688 (127.3.139.66) evicted. New config: { opid_index: 138 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:32.134438 5473 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet d14545ba1e474960909a1b140b6a315c with cas_config_opid_index 137: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:58:32.138003 5486 catalog_manager.cc:5484] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d reported cstate change: config changed from index 137 to 138, VOTER eda55a8566c447cfa749457825b70688 (127.3.139.66) evicted. New cstate: current_term: 1 leader_uuid: "d5bc7863cd294f77a932c4a40f98b79d" committed_config { opid_index: 138 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:32.141536 5740 tablet_service.cc:1514] Processing DeleteTablet for tablet d14545ba1e474960909a1b140b6a315c with delete_type TABLET_DATA_TOMBSTONED (TS eda55a8566c447cfa749457825b70688 not found in new config with opid_index 138) from {username='slave'} at 127.0.0.1:41882
I20240502 07:58:32.143934 5890 raft_consensus.cc:2932] T d14545ba1e474960909a1b140b6a315c P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Committing config change with OpId 1.138: config changed from index 137 to 138, VOTER eda55a8566c447cfa749457825b70688 (127.3.139.66) evicted. New config: { opid_index: 138 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:32.145804 7774 tablet_replica.cc:330] T d14545ba1e474960909a1b140b6a315c P eda55a8566c447cfa749457825b70688: stopping tablet replica
I20240502 07:58:32.146185 7774 raft_consensus.cc:2229] T d14545ba1e474960909a1b140b6a315c P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:32.146346 7774 pending_rounds.cc:62] T d14545ba1e474960909a1b140b6a315c P eda55a8566c447cfa749457825b70688: Trying to abort 2 pending ops.
I20240502 07:58:32.146407 7774 pending_rounds.cc:68] T d14545ba1e474960909a1b140b6a315c P eda55a8566c447cfa749457825b70688: Aborting op as it isn't in flight: id { term: 1 index: 137 } timestamp: 7023151972485636096 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "d14545ba1e474960909a1b140b6a315c" old_config { opid_index: 100 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: true } } } new_config { opid_index: 137 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } } }
I20240502 07:58:32.146634 7774 raft_consensus.cc:2894] T d14545ba1e474960909a1b140b6a315c P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Aborting config change with OpId 1.137: Aborted: Op aborted
I20240502 07:58:32.146737 7774 raft_consensus.cc:2866] T d14545ba1e474960909a1b140b6a315c P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20240502 07:58:32.146811 7774 pending_rounds.cc:68] T d14545ba1e474960909a1b140b6a315c P eda55a8566c447cfa749457825b70688: Aborting op as it isn't in flight: id { term: 1 index: 136 } timestamp: 7023151972448116736 op_type: WRITE_OP write_request { tablet_id: "d14545ba1e474960909a1b140b6a315c" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\007\000\257\031$=w\253\"\255\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000" indirect_data: "<redacted>""11f3209ae1c310fc" } external_consistency_mode: CLIENT_PROPAGATED propagated_timestamp: 7023151972385976320 authz_token { token_data: "\010\313\221\315\261\006\"3\n\005slave\022*\n e1e32c7b8b8c4c05b2f82c3195606811\020\001\030\001 \001(\001" signature: "<redacted>""^\370\2521\"\273\245\216\303*8k\225!\264\232\035-V\351\341G\361\343\277?\214<\276\223\2705`\022\1776R\264\310\300L\312\302\306*\362\252\331\353\303u\231\252\256E\357\306\321-=\322\343\247h" signing_key_seq_num: 0 } } request_id { client_id: "40df30120f8c4f508428a71cda74c245" seq_no: 403 first_incomplete_seq_no: 403 attempt_no: 0 }
I20240502 07:58:32.147253 7774 raft_consensus.cc:2258] T d14545ba1e474960909a1b140b6a315c P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:32.150684 5626 raft_consensus.cc:2932] T d14545ba1e474960909a1b140b6a315c P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Committing config change with OpId 1.138: config changed from index 137 to 138, VOTER eda55a8566c447cfa749457825b70688 (127.3.139.66) evicted. New config: { opid_index: 138 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:32.157886 7774 ts_tablet_manager.cc:1875] T d14545ba1e474960909a1b140b6a315c P eda55a8566c447cfa749457825b70688: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:58:32.160557 7774 ts_tablet_manager.cc:1888] T d14545ba1e474960909a1b140b6a315c P eda55a8566c447cfa749457825b70688: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.137
I20240502 07:58:32.160676 7774 log.cc:1196] T d14545ba1e474960909a1b140b6a315c P eda55a8566c447cfa749457825b70688: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal/wals/d14545ba1e474960909a1b140b6a315c
I20240502 07:58:32.161520 5473 catalog_manager.cc:4830] TS eda55a8566c447cfa749457825b70688 (127.3.139.66:39925): tablet d14545ba1e474960909a1b140b6a315c (table rebalance_test_table_0 [id=e1e32c7b8b8c4c05b2f82c3195606811]) successfully deleted
I20240502 07:58:32.163523 6153 raft_consensus.cc:2932] T d14545ba1e474960909a1b140b6a315c P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.138: config changed from index 137 to 138, VOTER eda55a8566c447cfa749457825b70688 (127.3.139.66) evicted. New config: { opid_index: 138 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:32.172592 7044 raft_consensus.cc:2932] T d14545ba1e474960909a1b140b6a315c P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Committing config change with OpId 1.138: config changed from index 137 to 138, VOTER eda55a8566c447cfa749457825b70688 (127.3.139.66) evicted. New config: { opid_index: 138 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } }
I20240502 07:58:32.562711 5872 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:32.568250 5608 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:32.601675 6002 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:58:32.607116 6132 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:32.653498 7028 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:32.691910 6263 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:58:32.742676 5740 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:33.181768 5628 consensus_queue.cc:237] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 275, Committed index: 275, Last appended: 1.276, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 277 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: NON_VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: true } }
I20240502 07:58:33.195325 5757 raft_consensus.cc:1261] T 96dcb06388374514951f24393f288eca P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 276. Preceding OpId from leader: term: 1 index: 277. (index mismatch)
I20240502 07:58:33.195894 6020 raft_consensus.cc:1261] T 96dcb06388374514951f24393f288eca P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 276. Preceding OpId from leader: term: 1 index: 277. (index mismatch)
I20240502 07:58:33.196116 7325 consensus_queue.cc:1035] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 277, Last known committed idx: 276, Time since last communication: 0.000s
W20240502 07:58:33.197793 5564 consensus_peers.cc:485] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 -> Peer 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67:37141): Couldn't send request to peer 89bf5c0a6b2a4d949851089446f17b96. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 96dcb06388374514951f24393f288eca. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:33.202800 6149 raft_consensus.cc:1261] T 96dcb06388374514951f24393f288eca P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 275. Preceding OpId from leader: term: 1 index: 277. (index mismatch)
I20240502 07:58:33.206185 7599 consensus_queue.cc:1035] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 277, Last known committed idx: 275, Time since last communication: 0.000s
I20240502 07:58:33.211724 7044 raft_consensus.cc:1261] T 96dcb06388374514951f24393f288eca P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 276. Preceding OpId from leader: term: 1 index: 277. (index mismatch)
I20240502 07:58:33.214062 7599 consensus_queue.cc:1035] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 277, Last known committed idx: 275, Time since last communication: 0.000s
I20240502 07:58:33.223702 7482 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [term 1 LEADER]: Committing config change with OpId 1.277: config changed from index 133 to 277, NON_VOTER 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67) added. New config: { opid_index: 277 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: NON_VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: true } } }
I20240502 07:58:33.226562 5488 catalog_manager.cc:5484] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 reported cstate change: config changed from index 133 to 277, NON_VOTER 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67) added. New cstate: current_term: 1 leader_uuid: "91e82003a656442189a329b29846a841" committed_config { opid_index: 277 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: NON_VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:33.239467 6020 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Committing config change with OpId 1.277: config changed from index 133 to 277, NON_VOTER 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67) added. New config: { opid_index: 277 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: NON_VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: true } } }
I20240502 07:58:33.259409 7599 consensus_queue.cc:1035] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 277, Last known committed idx: 275, Time since last communication: 0.000s
I20240502 07:58:33.263532 7044 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Committing config change with OpId 1.277: config changed from index 133 to 277, NON_VOTER 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67) added. New config: { opid_index: 277 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: NON_VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: true } } }
I20240502 07:58:33.264825 6153 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.277: config changed from index 133 to 277, NON_VOTER 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67) added. New config: { opid_index: 277 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: NON_VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: true } } }
I20240502 07:58:33.271701 5761 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Committing config change with OpId 1.277: config changed from index 133 to 277, NON_VOTER 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67) added. New config: { opid_index: 277 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: NON_VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: true } } }
I20240502 07:58:33.339423 6283 consensus_queue.cc:237] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 155, Committed index: 155, Last appended: 1.156, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 157 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: NON_VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: true } }
W20240502 07:58:33.341670 6219 consensus_peers.cc:485] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb -> Peer eda55a8566c447cfa749457825b70688 (127.3.139.66:39925): Couldn't send request to peer eda55a8566c447cfa749457825b70688. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d41a2cb7b69e40d9a681384e5ad0a451. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:33.342247 5890 raft_consensus.cc:1261] T d41a2cb7b69e40d9a681384e5ad0a451 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 155. Preceding OpId from leader: term: 1 index: 157. (index mismatch)
I20240502 07:58:33.342391 5626 raft_consensus.cc:1261] T d41a2cb7b69e40d9a681384e5ad0a451 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 155. Preceding OpId from leader: term: 1 index: 157. (index mismatch)
I20240502 07:58:33.343282 7253 consensus_queue.cc:1035] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 157, Last known committed idx: 155, Time since last communication: 0.000s
I20240502 07:58:33.345044 7253 consensus_queue.cc:1035] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 157, Last known committed idx: 155, Time since last communication: 0.000s
I20240502 07:58:33.349462 7254 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 LEADER]: Committing config change with OpId 1.157: config changed from index 67 to 157, NON_VOTER eda55a8566c447cfa749457825b70688 (127.3.139.66) added. New config: { opid_index: 157 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: NON_VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: true } } }
I20240502 07:58:33.351387 5890 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Committing config change with OpId 1.157: config changed from index 67 to 157, NON_VOTER eda55a8566c447cfa749457825b70688 (127.3.139.66) added. New config: { opid_index: 157 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: NON_VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: true } } }
I20240502 07:58:33.351514 5627 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Committing config change with OpId 1.157: config changed from index 67 to 157, NON_VOTER eda55a8566c447cfa749457825b70688 (127.3.139.66) added. New config: { opid_index: 157 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: NON_VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: true } } }
I20240502 07:58:33.354131 5486 catalog_manager.cc:5484] T d41a2cb7b69e40d9a681384e5ad0a451 P 89bf5c0a6b2a4d949851089446f17b96 reported cstate change: config changed from index 67 to 157, NON_VOTER eda55a8566c447cfa749457825b70688 (127.3.139.66) added. New cstate: current_term: 1 leader_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" committed_config { opid_index: 157 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: NON_VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: true } } }
I20240502 07:58:33.402920 6149 raft_consensus.cc:1261] T d41a2cb7b69e40d9a681384e5ad0a451 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 155. Preceding OpId from leader: term: 1 index: 157. (index mismatch)
I20240502 07:58:33.403426 7044 raft_consensus.cc:1261] T d41a2cb7b69e40d9a681384e5ad0a451 P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 155. Preceding OpId from leader: term: 1 index: 157. (index mismatch)
I20240502 07:58:33.405076 7610 consensus_queue.cc:1035] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 157, Last known committed idx: 154, Time since last communication: 0.000s
I20240502 07:58:33.408221 7610 consensus_queue.cc:1035] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 157, Last known committed idx: 154, Time since last communication: 0.000s
I20240502 07:58:33.418233 6149 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.157: config changed from index 67 to 157, NON_VOTER eda55a8566c447cfa749457825b70688 (127.3.139.66) added. New config: { opid_index: 157 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: NON_VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: true } } }
I20240502 07:58:33.429533 7045 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Committing config change with OpId 1.157: config changed from index 67 to 157, NON_VOTER eda55a8566c447cfa749457825b70688 (127.3.139.66) added. New config: { opid_index: 157 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: NON_VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: true } } }
I20240502 07:58:33.766389 7878 ts_tablet_manager.cc:902] T 96dcb06388374514951f24393f288eca P 89bf5c0a6b2a4d949851089446f17b96: Initiating tablet copy from peer 91e82003a656442189a329b29846a841 (127.3.139.65:43429)
I20240502 07:58:33.772542 7879 ts_tablet_manager.cc:902] T d41a2cb7b69e40d9a681384e5ad0a451 P eda55a8566c447cfa749457825b70688: Initiating tablet copy from peer d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70:41047)
I20240502 07:58:33.773325 7879 tablet_copy_client.cc:308] T d41a2cb7b69e40d9a681384e5ad0a451 P eda55a8566c447cfa749457825b70688: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.70:41047
I20240502 07:58:33.796331 7878 tablet_copy_client.cc:308] T 96dcb06388374514951f24393f288eca P 89bf5c0a6b2a4d949851089446f17b96: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.65:43429
I20240502 07:58:33.858477 5650 tablet_copy_service.cc:140] P 91e82003a656442189a329b29846a841: Received BeginTabletCopySession request for tablet 96dcb06388374514951f24393f288eca from peer 89bf5c0a6b2a4d949851089446f17b96 ({username='slave'} at 127.3.139.67:38019)
I20240502 07:58:33.858584 5650 tablet_copy_service.cc:161] P 91e82003a656442189a329b29846a841: Beginning new tablet copy session on tablet 96dcb06388374514951f24393f288eca from peer 89bf5c0a6b2a4d949851089446f17b96 at {username='slave'} at 127.3.139.67:38019: session id = 89bf5c0a6b2a4d949851089446f17b96-96dcb06388374514951f24393f288eca
I20240502 07:58:33.859557 5650 tablet_copy_source_session.cc:205] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:33.862474 6303 tablet_copy_service.cc:140] P d58f6d1ff8cf472f8a9edf3700e64bfb: Received BeginTabletCopySession request for tablet d41a2cb7b69e40d9a681384e5ad0a451 from peer eda55a8566c447cfa749457825b70688 ({username='slave'} at 127.3.139.66:45365)
I20240502 07:58:33.862509 7878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96dcb06388374514951f24393f288eca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:33.862573 6303 tablet_copy_service.cc:161] P d58f6d1ff8cf472f8a9edf3700e64bfb: Beginning new tablet copy session on tablet d41a2cb7b69e40d9a681384e5ad0a451 from peer eda55a8566c447cfa749457825b70688 at {username='slave'} at 127.3.139.66:45365: session id = eda55a8566c447cfa749457825b70688-d41a2cb7b69e40d9a681384e5ad0a451
I20240502 07:58:33.863539 6303 tablet_copy_source_session.cc:205] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:33.867620 7878 tablet_copy_client.cc:783] T 96dcb06388374514951f24393f288eca P 89bf5c0a6b2a4d949851089446f17b96: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:33.869269 7878 tablet_copy_client.cc:647] T 96dcb06388374514951f24393f288eca P 89bf5c0a6b2a4d949851089446f17b96: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:33.877990 7879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d41a2cb7b69e40d9a681384e5ad0a451. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:33.880319 7879 tablet_copy_client.cc:783] T d41a2cb7b69e40d9a681384e5ad0a451 P eda55a8566c447cfa749457825b70688: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:33.880489 7879 tablet_copy_client.cc:647] T d41a2cb7b69e40d9a681384e5ad0a451 P eda55a8566c447cfa749457825b70688: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:33.900863 7878 tablet_copy_client.cc:523] T 96dcb06388374514951f24393f288eca P 89bf5c0a6b2a4d949851089446f17b96: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:33.903429 7878 tablet_bootstrap.cc:492] T 96dcb06388374514951f24393f288eca P 89bf5c0a6b2a4d949851089446f17b96: Bootstrap starting.
W20240502 07:58:33.923715 6219 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.139.65:43429, user_credentials={real_user=slave}} blocked reactor thread for 46664us
I20240502 07:58:34.004391 7879 tablet_copy_client.cc:523] T d41a2cb7b69e40d9a681384e5ad0a451 P eda55a8566c447cfa749457825b70688: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:34.010162 7879 tablet_bootstrap.cc:492] T d41a2cb7b69e40d9a681384e5ad0a451 P eda55a8566c447cfa749457825b70688: Bootstrap starting.
W20240502 07:58:34.520391 5695 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.139.67:37141, user_credentials={real_user=slave}} blocked reactor thread for 59569.4us
W20240502 07:58:34.674494 5957 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.139.66:39925, user_credentials={real_user=slave}} blocked reactor thread for 51349.6us
I20240502 07:58:34.692481 7879 tablet_bootstrap.cc:492] T d41a2cb7b69e40d9a681384e5ad0a451 P eda55a8566c447cfa749457825b70688: Bootstrap replayed 1/1 log segments. Stats: ops{read=165 overwritten=0 applied=165 ignored=0} inserts{seen=847 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:58:34.692840 7879 tablet_bootstrap.cc:492] T d41a2cb7b69e40d9a681384e5ad0a451 P eda55a8566c447cfa749457825b70688: Bootstrap complete.
I20240502 07:58:34.693171 7879 ts_tablet_manager.cc:1367] T d41a2cb7b69e40d9a681384e5ad0a451 P eda55a8566c447cfa749457825b70688: Time spent bootstrapping tablet: real 0.683s user 0.114s sys 0.025s
I20240502 07:58:34.693631 7879 raft_consensus.cc:348] T d41a2cb7b69e40d9a681384e5ad0a451 P eda55a8566c447cfa749457825b70688 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 157 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: NON_VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: true } }
I20240502 07:58:34.693773 7879 raft_consensus.cc:729] T d41a2cb7b69e40d9a681384e5ad0a451 P eda55a8566c447cfa749457825b70688 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: eda55a8566c447cfa749457825b70688, State: Initialized, Role: LEARNER
I20240502 07:58:34.694278 7879 consensus_queue.cc:260] T d41a2cb7b69e40d9a681384e5ad0a451 P eda55a8566c447cfa749457825b70688 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 165, Last appended: 1.165, Last appended by leader: 165, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 157 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: NON_VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: true } }
I20240502 07:58:34.696293 7879 ts_tablet_manager.cc:1398] T d41a2cb7b69e40d9a681384e5ad0a451 P eda55a8566c447cfa749457825b70688: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:58:34.801388 6303 tablet_copy_service.cc:342] P d58f6d1ff8cf472f8a9edf3700e64bfb: Request end of tablet copy session eda55a8566c447cfa749457825b70688-d41a2cb7b69e40d9a681384e5ad0a451 received from {username='slave'} at 127.3.139.66:45365
I20240502 07:58:34.801497 6303 tablet_copy_service.cc:433] P d58f6d1ff8cf472f8a9edf3700e64bfb: ending tablet copy session eda55a8566c447cfa749457825b70688-d41a2cb7b69e40d9a681384e5ad0a451 on tablet d41a2cb7b69e40d9a681384e5ad0a451 with peer eda55a8566c447cfa749457825b70688
I20240502 07:58:34.802243 5757 raft_consensus.cc:1203] T d41a2cb7b69e40d9a681384e5ad0a451 P eda55a8566c447cfa749457825b70688 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.156->[1.157-1.172] Dedup: 1.165->[1.166-1.172]
I20240502 07:58:34.811236 7878 tablet_bootstrap.cc:492] T 96dcb06388374514951f24393f288eca P 89bf5c0a6b2a4d949851089446f17b96: Bootstrap replayed 1/1 log segments. Stats: ops{read=297 overwritten=0 applied=296 ignored=0} inserts{seen=978 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240502 07:58:34.811861 7878 tablet_bootstrap.cc:492] T 96dcb06388374514951f24393f288eca P 89bf5c0a6b2a4d949851089446f17b96: Bootstrap complete.
I20240502 07:58:34.812130 7878 ts_tablet_manager.cc:1367] T 96dcb06388374514951f24393f288eca P 89bf5c0a6b2a4d949851089446f17b96: Time spent bootstrapping tablet: real 0.909s user 0.199s sys 0.033s
I20240502 07:58:34.812639 7878 raft_consensus.cc:348] T 96dcb06388374514951f24393f288eca P 89bf5c0a6b2a4d949851089446f17b96 [term 1 LEARNER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: 277 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: NON_VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: true } }
I20240502 07:58:34.813051 7878 raft_consensus.cc:729] T 96dcb06388374514951f24393f288eca P 89bf5c0a6b2a4d949851089446f17b96 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 89bf5c0a6b2a4d949851089446f17b96, State: Initialized, Role: LEARNER
I20240502 07:58:34.813230 7878 consensus_queue.cc:260] T 96dcb06388374514951f24393f288eca P 89bf5c0a6b2a4d949851089446f17b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 296, Last appended: 1.297, Last appended by leader: 297, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 277 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: NON_VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: true } }
I20240502 07:58:34.816164 7878 ts_tablet_manager.cc:1398] T 96dcb06388374514951f24393f288eca P 89bf5c0a6b2a4d949851089446f17b96: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:58:34.818009 5650 tablet_copy_service.cc:342] P 91e82003a656442189a329b29846a841: Request end of tablet copy session 89bf5c0a6b2a4d949851089446f17b96-96dcb06388374514951f24393f288eca received from {username='slave'} at 127.3.139.67:38019
I20240502 07:58:34.818087 5650 tablet_copy_service.cc:433] P 91e82003a656442189a329b29846a841: ending tablet copy session 89bf5c0a6b2a4d949851089446f17b96-96dcb06388374514951f24393f288eca on tablet 96dcb06388374514951f24393f288eca with peer 89bf5c0a6b2a4d949851089446f17b96
I20240502 07:58:34.879753 6674 raft_consensus.cc:1050] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb: attempting to promote NON_VOTER eda55a8566c447cfa749457825b70688 to VOTER
I20240502 07:58:34.880369 6674 consensus_queue.cc:237] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 174, Committed index: 174, Last appended: 1.174, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 175 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: false } }
I20240502 07:58:34.882225 7048 raft_consensus.cc:1261] T d41a2cb7b69e40d9a681384e5ad0a451 P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 174. Preceding OpId from leader: term: 1 index: 175. (index mismatch)
I20240502 07:58:34.882725 6151 raft_consensus.cc:1261] T d41a2cb7b69e40d9a681384e5ad0a451 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 174. Preceding OpId from leader: term: 1 index: 175. (index mismatch)
I20240502 07:58:34.883107 6674 consensus_queue.cc:1035] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 175, Last known committed idx: 174, Time since last communication: 0.000s
I20240502 07:58:34.883399 6674 consensus_queue.cc:1035] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 175, Last known committed idx: 174, Time since last communication: 0.000s
I20240502 07:58:34.892170 5759 raft_consensus.cc:1261] T d41a2cb7b69e40d9a681384e5ad0a451 P eda55a8566c447cfa749457825b70688 [term 1 LEARNER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 174. Preceding OpId from leader: term: 1 index: 175. (index mismatch)
I20240502 07:58:34.892951 5627 raft_consensus.cc:1261] T d41a2cb7b69e40d9a681384e5ad0a451 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 174. Preceding OpId from leader: term: 1 index: 175. (index mismatch)
I20240502 07:58:34.893534 5891 raft_consensus.cc:1261] T d41a2cb7b69e40d9a681384e5ad0a451 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 174. Preceding OpId from leader: term: 1 index: 176. (index mismatch)
I20240502 07:58:34.897486 7868 consensus_queue.cc:1035] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 175, Last known committed idx: 174, Time since last communication: 0.000s
I20240502 07:58:34.898449 7868 consensus_queue.cc:1035] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 175, Last known committed idx: 174, Time since last communication: 0.000s
I20240502 07:58:34.903183 7868 consensus_queue.cc:1035] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 175, Last known committed idx: 174, Time since last communication: 0.000s
I20240502 07:58:34.915670 6674 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 LEADER]: Committing config change with OpId 1.175: config changed from index 157 to 175, eda55a8566c447cfa749457825b70688 (127.3.139.66) changed from NON_VOTER to VOTER. New config: { opid_index: 175 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: false } } }
I20240502 07:58:34.919461 5480 catalog_manager.cc:5484] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb reported cstate change: config changed from index 157 to 175, eda55a8566c447cfa749457825b70688 (127.3.139.66) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" committed_config { opid_index: 175 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:34.923650 7046 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Committing config change with OpId 1.175: config changed from index 157 to 175, eda55a8566c447cfa749457825b70688 (127.3.139.66) changed from NON_VOTER to VOTER. New config: { opid_index: 175 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: false } } }
I20240502 07:58:34.936440 5627 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Committing config change with OpId 1.175: config changed from index 157 to 175, eda55a8566c447cfa749457825b70688 (127.3.139.66) changed from NON_VOTER to VOTER. New config: { opid_index: 175 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: false } } }
I20240502 07:58:34.936547 5888 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Committing config change with OpId 1.175: config changed from index 157 to 175, eda55a8566c447cfa749457825b70688 (127.3.139.66) changed from NON_VOTER to VOTER. New config: { opid_index: 175 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: false } } }
I20240502 07:58:34.942332 5759 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Committing config change with OpId 1.175: config changed from index 157 to 175, eda55a8566c447cfa749457825b70688 (127.3.139.66) changed from NON_VOTER to VOTER. New config: { opid_index: 175 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: false } } }
I20240502 07:58:34.947659 6148 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.175: config changed from index 157 to 175, eda55a8566c447cfa749457825b70688 (127.3.139.66) changed from NON_VOTER to VOTER. New config: { opid_index: 175 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: false } } }
I20240502 07:58:35.172001 6282 consensus_queue.cc:237] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 181, Committed index: 181, Last appended: 1.181, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 182 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: false } }
I20240502 07:58:35.176327 7046 raft_consensus.cc:1261] T d41a2cb7b69e40d9a681384e5ad0a451 P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 181. Preceding OpId from leader: term: 1 index: 182. (index mismatch)
I20240502 07:58:35.176990 7868 consensus_queue.cc:1035] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 182, Last known committed idx: 181, Time since last communication: 0.000s
I20240502 07:58:35.177112 6150 raft_consensus.cc:1261] T d41a2cb7b69e40d9a681384e5ad0a451 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 181. Preceding OpId from leader: term: 1 index: 182. (index mismatch)
I20240502 07:58:35.177491 7868 consensus_queue.cc:1035] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 182, Last known committed idx: 181, Time since last communication: 0.000s
I20240502 07:58:35.183362 7839 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 LEADER]: Committing config change with OpId 1.182: config changed from index 175 to 182, VOTER 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67) evicted. New config: { opid_index: 182 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: false } } }
I20240502 07:58:35.187176 5474 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet d41a2cb7b69e40d9a681384e5ad0a451 with cas_config_opid_index 175: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:58:35.187893 5482 catalog_manager.cc:5484] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb reported cstate change: config changed from index 175 to 182, VOTER 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67) evicted. New cstate: current_term: 1 leader_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" committed_config { opid_index: 182 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:35.189102 6153 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.182: config changed from index 175 to 182, VOTER 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67) evicted. New config: { opid_index: 182 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: false } } }
I20240502 07:58:35.192093 7046 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Committing config change with OpId 1.182: config changed from index 175 to 182, VOTER 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67) evicted. New config: { opid_index: 182 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: false } } }
I20240502 07:58:35.193876 5759 raft_consensus.cc:1261] T d41a2cb7b69e40d9a681384e5ad0a451 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 181. Preceding OpId from leader: term: 1 index: 182. (index mismatch)
I20240502 07:58:35.201696 5627 raft_consensus.cc:1261] T d41a2cb7b69e40d9a681384e5ad0a451 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 181. Preceding OpId from leader: term: 1 index: 182. (index mismatch)
I20240502 07:58:35.206688 6674 consensus_queue.cc:1035] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 182, Last known committed idx: 181, Time since last communication: 0.000s
I20240502 07:58:35.207653 7253 consensus_queue.cc:1035] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 182, Last known committed idx: 181, Time since last communication: 0.000s
I20240502 07:58:35.211639 5627 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Committing config change with OpId 1.182: config changed from index 175 to 182, VOTER 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67) evicted. New config: { opid_index: 182 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: false } } }
I20240502 07:58:35.217214 5759 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Committing config change with OpId 1.182: config changed from index 175 to 182, VOTER 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67) evicted. New config: { opid_index: 182 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: false } } }
I20240502 07:58:35.227624 5872 tablet_service.cc:1514] Processing DeleteTablet for tablet d41a2cb7b69e40d9a681384e5ad0a451 with delete_type TABLET_DATA_TOMBSTONED (TS 89bf5c0a6b2a4d949851089446f17b96 not found in new config with opid_index 182) from {username='slave'} at 127.0.0.1:42126
I20240502 07:58:35.246867 7943 tablet_replica.cc:330] T d41a2cb7b69e40d9a681384e5ad0a451 P 89bf5c0a6b2a4d949851089446f17b96: stopping tablet replica
I20240502 07:58:35.246999 7943 raft_consensus.cc:2229] T d41a2cb7b69e40d9a681384e5ad0a451 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:35.247133 7943 raft_consensus.cc:2258] T d41a2cb7b69e40d9a681384e5ad0a451 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:35.257510 5887 raft_consensus.cc:1203] T 96dcb06388374514951f24393f288eca P 89bf5c0a6b2a4d949851089446f17b96 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.276->[1.277-1.336] Dedup: 1.297->[1.298-1.336]
I20240502 07:58:35.281260 7943 ts_tablet_manager.cc:1875] T d41a2cb7b69e40d9a681384e5ad0a451 P 89bf5c0a6b2a4d949851089446f17b96: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:58:35.301842 7943 ts_tablet_manager.cc:1888] T d41a2cb7b69e40d9a681384e5ad0a451 P 89bf5c0a6b2a4d949851089446f17b96: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.181
I20240502 07:58:35.303555 7943 log.cc:1196] T d41a2cb7b69e40d9a681384e5ad0a451 P 89bf5c0a6b2a4d949851089446f17b96: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal/wals/d41a2cb7b69e40d9a681384e5ad0a451
I20240502 07:58:35.304195 5473 catalog_manager.cc:4830] TS 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67:37141): tablet d41a2cb7b69e40d9a681384e5ad0a451 (table rebalance_test_table_1 [id=6f5d44f6fbe544099b33ad94e35cee6e]) successfully deleted
I20240502 07:58:35.423466 7482 raft_consensus.cc:1050] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841: attempting to promote NON_VOTER 89bf5c0a6b2a4d949851089446f17b96 to VOTER
I20240502 07:58:35.424691 7482 consensus_queue.cc:237] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 339, Committed index: 339, Last appended: 1.340, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 341 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: false } }
I20240502 07:58:35.434309 7044 raft_consensus.cc:1261] T 96dcb06388374514951f24393f288eca P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 340. Preceding OpId from leader: term: 1 index: 341. (index mismatch)
I20240502 07:58:35.435561 7677 consensus_queue.cc:1035] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 341, Last known committed idx: 339, Time since last communication: 0.000s
I20240502 07:58:35.435752 6148 raft_consensus.cc:1261] T 96dcb06388374514951f24393f288eca P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 340. Preceding OpId from leader: term: 1 index: 341. (index mismatch)
I20240502 07:58:35.436990 5887 raft_consensus.cc:1261] T 96dcb06388374514951f24393f288eca P 89bf5c0a6b2a4d949851089446f17b96 [term 1 LEARNER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 340. Preceding OpId from leader: term: 1 index: 341. (index mismatch)
I20240502 07:58:35.438097 6022 raft_consensus.cc:1261] T 96dcb06388374514951f24393f288eca P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 339. Preceding OpId from leader: term: 1 index: 341. (index mismatch)
I20240502 07:58:35.439657 5761 raft_consensus.cc:1261] T 96dcb06388374514951f24393f288eca P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 340. Preceding OpId from leader: term: 1 index: 341. (index mismatch)
I20240502 07:58:35.439904 7599 consensus_queue.cc:1035] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 341, Last known committed idx: 339, Time since last communication: 0.000s
I20240502 07:58:35.445672 7599 consensus_queue.cc:1035] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 341, Last known committed idx: 339, Time since last communication: 0.000s
I20240502 07:58:35.450832 7599 consensus_queue.cc:1035] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 341, Last known committed idx: 339, Time since last communication: 0.000s
I20240502 07:58:35.452239 7599 consensus_queue.cc:1035] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 341, Last known committed idx: 339, Time since last communication: 0.000s
I20240502 07:58:35.470275 7325 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [term 1 LEADER]: Committing config change with OpId 1.341: config changed from index 277 to 341, 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67) changed from NON_VOTER to VOTER. New config: { opid_index: 341 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: false } } }
I20240502 07:58:35.473136 5484 catalog_manager.cc:5484] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 reported cstate change: config changed from index 277 to 341, 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "91e82003a656442189a329b29846a841" committed_config { opid_index: 341 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:35.474309 7044 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Committing config change with OpId 1.341: config changed from index 277 to 341, 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67) changed from NON_VOTER to VOTER. New config: { opid_index: 341 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: false } } }
I20240502 07:58:35.493937 6151 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.341: config changed from index 277 to 341, 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67) changed from NON_VOTER to VOTER. New config: { opid_index: 341 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: false } } }
I20240502 07:58:35.520027 6020 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Committing config change with OpId 1.341: config changed from index 277 to 341, 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67) changed from NON_VOTER to VOTER. New config: { opid_index: 341 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: false } } }
I20240502 07:58:35.523815 5891 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Committing config change with OpId 1.341: config changed from index 277 to 341, 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67) changed from NON_VOTER to VOTER. New config: { opid_index: 341 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: false } } }
I20240502 07:58:35.526249 5629 consensus_queue.cc:237] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 341, Committed index: 341, Last appended: 1.342, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 343 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: false } }
I20240502 07:58:35.530675 5757 raft_consensus.cc:1261] T 96dcb06388374514951f24393f288eca P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 341. Preceding OpId from leader: term: 1 index: 343. (index mismatch)
I20240502 07:58:35.533154 5891 raft_consensus.cc:1261] T 96dcb06388374514951f24393f288eca P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 342. Preceding OpId from leader: term: 1 index: 343. (index mismatch)
I20240502 07:58:35.535445 5761 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Committing config change with OpId 1.341: config changed from index 277 to 341, 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67) changed from NON_VOTER to VOTER. New config: { opid_index: 341 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: false } } }
I20240502 07:58:35.542459 7044 raft_consensus.cc:1261] T 96dcb06388374514951f24393f288eca P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 342. Preceding OpId from leader: term: 1 index: 343. (index mismatch)
I20240502 07:58:35.543596 7599 consensus_queue.cc:1035] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 343, Last known committed idx: 341, Time since last communication: 0.000s
I20240502 07:58:35.543958 6149 raft_consensus.cc:1261] T 96dcb06388374514951f24393f288eca P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 342. Preceding OpId from leader: term: 1 index: 343. (index mismatch)
I20240502 07:58:35.547884 7599 consensus_queue.cc:1035] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 343, Last known committed idx: 339, Time since last communication: 0.000s
I20240502 07:58:35.549702 7677 consensus_queue.cc:1035] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 343, Last known committed idx: 341, Time since last communication: 0.000s
I20240502 07:58:35.550541 7677 consensus_queue.cc:1035] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 343, Last known committed idx: 341, Time since last communication: 0.000s
I20240502 07:58:35.561686 5757 raft_consensus.cc:1203] T 96dcb06388374514951f24393f288eca P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.341->[1.342-1.343] Dedup: 1.342->[1.343-1.343]
I20240502 07:58:35.578346 6740 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [term 1 LEADER]: Committing config change with OpId 1.343: config changed from index 341 to 343, VOTER d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68) evicted. New config: { opid_index: 343 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: false } } }
I20240502 07:58:35.581750 5473 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 96dcb06388374514951f24393f288eca with cas_config_opid_index 341: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:58:35.582640 5484 catalog_manager.cc:5484] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 reported cstate change: config changed from index 341 to 343, VOTER d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68) evicted. New cstate: current_term: 1 leader_uuid: "91e82003a656442189a329b29846a841" committed_config { opid_index: 343 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:58:35.584759 6151 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.343: config changed from index 341 to 343, VOTER d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68) evicted. New config: { opid_index: 343 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: false } } }
I20240502 07:58:35.587517 6002 tablet_service.cc:1514] Processing DeleteTablet for tablet 96dcb06388374514951f24393f288eca with delete_type TABLET_DATA_TOMBSTONED (TS d5bc7863cd294f77a932c4a40f98b79d not found in new config with opid_index 343) from {username='slave'} at 127.0.0.1:39270
I20240502 07:58:35.588367 7044 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Committing config change with OpId 1.343: config changed from index 341 to 343, VOTER d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68) evicted. New config: { opid_index: 343 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: false } } }
I20240502 07:58:35.608757 5757 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Committing config change with OpId 1.343: config changed from index 341 to 343, VOTER d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68) evicted. New config: { opid_index: 343 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: false } } }
I20240502 07:58:35.635576 5891 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Committing config change with OpId 1.343: config changed from index 341 to 343, VOTER d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68) evicted. New config: { opid_index: 343 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: false } } }
I20240502 07:58:35.659469 7962 tablet_replica.cc:330] T 96dcb06388374514951f24393f288eca P d5bc7863cd294f77a932c4a40f98b79d: stopping tablet replica
I20240502 07:58:35.667045 7962 raft_consensus.cc:2229] T 96dcb06388374514951f24393f288eca P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:35.667611 7962 pending_rounds.cc:62] T 96dcb06388374514951f24393f288eca P d5bc7863cd294f77a932c4a40f98b79d: Trying to abort 1 pending ops.
I20240502 07:58:35.667672 7962 pending_rounds.cc:68] T 96dcb06388374514951f24393f288eca P d5bc7863cd294f77a932c4a40f98b79d: Aborting op as it isn't in flight: id { term: 1 index: 342 } timestamp: 7023151986651803648 op_type: WRITE_OP write_request { tablet_id: "96dcb06388374514951f24393f288eca" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\007\000\355\356E\001|~@\025\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000" indirect_data: "<redacted>""0e696614535964e0" } external_consistency_mode: CLIENT_PROPAGATED propagated_timestamp: 7023151986603282432 authz_token { token_data: "\010\313\221\315\261\006\"3\n\005slave\022*\n d0a0bf64867c4915ad797ab2bcd7101f\020\001\030\001 \001(\001" signature: "<redacted>""\271\255\033.\277\314\302\260\006\013\236@\273u\027\224\264\203\227&\204}\313Px\014\371M5\271\346\343\334\024k\270\304\236q\250V\225UW\212b\326\363 \211\243\314\214\261\236\004\365\240\330\376\371\352\226\033" signing_key_seq_num: 0 } } request_id { client_id: "05cd43938067401cb649a0c9aede8230" seq_no: 925 first_incomplete_seq_no: 925 attempt_no: 0 }
I20240502 07:58:35.667922 7962 raft_consensus.cc:2258] T 96dcb06388374514951f24393f288eca P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:35.673249 7962 ts_tablet_manager.cc:1875] T 96dcb06388374514951f24393f288eca P d5bc7863cd294f77a932c4a40f98b79d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:58:35.695305 7962 ts_tablet_manager.cc:1888] T 96dcb06388374514951f24393f288eca P d5bc7863cd294f77a932c4a40f98b79d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.342
I20240502 07:58:35.695398 7962 log.cc:1196] T 96dcb06388374514951f24393f288eca P d5bc7863cd294f77a932c4a40f98b79d: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal/wals/96dcb06388374514951f24393f288eca
I20240502 07:58:35.695923 5473 catalog_manager.cc:4830] TS d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68:44287): tablet 96dcb06388374514951f24393f288eca (table rebalance_test_table_3 [id=d0a0bf64867c4915ad797ab2bcd7101f]) successfully deleted
I20240502 07:58:36.612866 7028 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:36.647017 5872 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:36.672415 5608 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:36.726365 6132 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:36.736400 6263 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:58:36.743335 5740 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
W20240502 07:58:36.781234 5957 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.139.66:39925, user_credentials={real_user=slave}} blocked reactor thread for 49131.9us
I20240502 07:58:36.851408 6002 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
W20240502 07:58:36.883908 5565 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.139.69:39399, user_credentials={real_user=slave}} blocked reactor thread for 44423.2us
W20240502 07:58:37.187907 5565 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.139.69:39399, user_credentials={real_user=slave}} blocked reactor thread for 50784us
W20240502 07:58:37.537061 6219 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.139.65:43429, user_credentials={real_user=slave}} blocked reactor thread for 52923.1us
I20240502 07:58:38.013706 5872 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:38.092748 6263 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:58:38.093842 5608 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:38.104493 7028 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:38.111690 6132 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:38.154816 6002 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:58:38.226559 5740 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:39.652086 5608 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:39.661316 6263 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:58:39.736020 5872 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:39.764452 6132 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:39.773108 6002 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:58:39.816440 7028 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:39.837576 5740 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
W20240502 07:58:40.608287 6219 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.139.67:37141, user_credentials={real_user=slave}} blocked reactor thread for 53020.8us
I20240502 07:58:40.761462 6263 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:58:40.831187 7028 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:40.832548 6132 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:40.839938 5740 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
W20240502 07:58:40.842823 5564 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.139.67:37141, user_credentials={real_user=slave}} blocked reactor thread for 63895.4us
I20240502 07:58:40.872167 6002 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:58:40.903399 5872 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
W20240502 07:58:40.916576 6219 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.139.68:44287, user_credentials={real_user=slave}} blocked reactor thread for 61701.1us
I20240502 07:58:41.018795 5608 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:41.365718 5608 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:41.367893 6132 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:41.370057 6002 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:58:41.373864 6263 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:58:41.376794 7028 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:41.382470 5872 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:41.399081 5740 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
c5d1643b3f5747ccb4514a400dcc82b4 | 127.3.139.126:34225 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
allow_unsafe_replication_factor | true | runtime,unsafe | all 1 server(s) checked
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.3.139.84:41319 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
27e3fafad7a24bcda362d2197be87017 | 127.3.139.71:35341 | HEALTHY | <none> | 0 | 0
89bf5c0a6b2a4d949851089446f17b96 | 127.3.139.67:37141 | HEALTHY | <none> | 1 | 0
91e82003a656442189a329b29846a841 | 127.3.139.65:43429 | HEALTHY | <none> | 3 | 0
d58f6d1ff8cf472f8a9edf3700e64bfb | 127.3.139.70:41047 | HEALTHY | <none> | 4 | 0
d5bc7863cd294f77a932c4a40f98b79d | 127.3.139.68:44287 | HEALTHY | <none> | 4 | 0
e020e7ffe8ce423a8bdefec667161217 | 127.3.139.69:39399 | HEALTHY | <none> | 1 | 0
eda55a8566c447cfa749457825b70688 | 127.3.139.66:39925 | HEALTHY | <none> | 3 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 7
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_server_key_size | 768 | experimental | all 7 server(s) checked
local_ip_for_outbound_sockets | 127.3.139.65 | experimental | 127.3.139.65:43429
local_ip_for_outbound_sockets | 127.3.139.66 | experimental | 127.3.139.66:39925
local_ip_for_outbound_sockets | 127.3.139.67 | experimental | 127.3.139.67:37141
local_ip_for_outbound_sockets | 127.3.139.68 | experimental | 127.3.139.68:44287
local_ip_for_outbound_sockets | 127.3.139.69 | experimental | 127.3.139.69:39399
local_ip_for_outbound_sockets | 127.3.139.70 | experimental | 127.3.139.70:41047
local_ip_for_outbound_sockets | 127.3.139.71 | experimental | 127.3.139.71:35341
never_fsync | true | unsafe,advanced | all 7 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 7 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 7 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 7 server(s) checked
server_dump_info_format | pb | hidden | all 7 server(s) checked
server_dump_info_path | /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden | 127.3.139.65:43429
server_dump_info_path | /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.3.139.66:39925
server_dump_info_path | /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.3.139.67:37141
server_dump_info_path | /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.3.139.68:44287
server_dump_info_path | /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.3.139.69:39399
server_dump_info_path | /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.3.139.70:41047
server_dump_info_path | /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/data/info.pb | hidden | 127.3.139.71:35341
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.3.139.84:41319 | all 7 server(s) checked
time_source | builtin | all 7 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 8 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
------------------------+----+---------+---------------+---------+------------+------------------+-------------
TestTable | 5 | HEALTHY | 1 | 1 | 0 | 0 | 0
rebalance_test_table_0 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_1 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_2 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_3 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_4 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 11
First Quartile | 11
Median | 11
Third Quartile | 12
Maximum | 12
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 7
Tables | 6
Tablets | 16
Replicas | 80
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20240502 07:58:41.432307 3629 log_verifier.cc:126] Checking tablet 07d53332bdd34233b3653756770dc9cf
I20240502 07:58:41.538163 3629 log_verifier.cc:177] Verified matching terms for 200 ops in tablet 07d53332bdd34233b3653756770dc9cf
I20240502 07:58:41.538339 3629 log_verifier.cc:126] Checking tablet 18a448dbe2af493abefbc728a8f85186
I20240502 07:58:41.685354 3629 log_verifier.cc:177] Verified matching terms for 311 ops in tablet 18a448dbe2af493abefbc728a8f85186
I20240502 07:58:41.685571 3629 log_verifier.cc:126] Checking tablet 2a3919f144ed4a27b03e7c4a251499d4
I20240502 07:58:41.875026 3629 log_verifier.cc:177] Verified matching terms for 426 ops in tablet 2a3919f144ed4a27b03e7c4a251499d4
I20240502 07:58:41.875341 3629 log_verifier.cc:126] Checking tablet 5b87554a301044caac956bc5c7f137e8
I20240502 07:58:42.009042 3629 log_verifier.cc:177] Verified matching terms for 296 ops in tablet 5b87554a301044caac956bc5c7f137e8
I20240502 07:58:42.009246 3629 log_verifier.cc:126] Checking tablet 5c8c7437c85943ba9113595c17bf6136
I20240502 07:58:42.142495 3629 log_verifier.cc:177] Verified matching terms for 289 ops in tablet 5c8c7437c85943ba9113595c17bf6136
I20240502 07:58:42.142683 3629 log_verifier.cc:126] Checking tablet 635b0a1a0fd64eeeac70025014c9ab81
I20240502 07:58:42.281539 3629 log_verifier.cc:177] Verified matching terms for 292 ops in tablet 635b0a1a0fd64eeeac70025014c9ab81
I20240502 07:58:42.281728 3629 log_verifier.cc:126] Checking tablet 7001a52697f1417bb84a0e3ed673ed0c
I20240502 07:58:42.426043 3629 log_verifier.cc:177] Verified matching terms for 312 ops in tablet 7001a52697f1417bb84a0e3ed673ed0c
I20240502 07:58:42.426244 3629 log_verifier.cc:126] Checking tablet 7f3d5d24bc164883adb1a11552500ce6
I20240502 07:58:42.518316 3629 log_verifier.cc:177] Verified matching terms for 162 ops in tablet 7f3d5d24bc164883adb1a11552500ce6
I20240502 07:58:42.518451 3629 log_verifier.cc:126] Checking tablet 96dcb06388374514951f24393f288eca
I20240502 07:58:42.723138 3629 log_verifier.cc:177] Verified matching terms for 500 ops in tablet 96dcb06388374514951f24393f288eca
I20240502 07:58:42.723423 3629 log_verifier.cc:126] Checking tablet acf2ff61e8ff450ab7fca71fb4191dd6
I20240502 07:58:42.867935 3629 log_verifier.cc:177] Verified matching terms for 316 ops in tablet acf2ff61e8ff450ab7fca71fb4191dd6
I20240502 07:58:42.868127 3629 log_verifier.cc:126] Checking tablet bc41794fb8c545149694aa84dde1cdb0
I20240502 07:58:43.051034 3629 log_verifier.cc:177] Verified matching terms for 448 ops in tablet bc41794fb8c545149694aa84dde1cdb0
I20240502 07:58:43.051339 3629 log_verifier.cc:126] Checking tablet cd070443095547d1bf82a6c4a7666b86
I20240502 07:58:43.085732 3629 log_verifier.cc:177] Verified matching terms for 4 ops in tablet cd070443095547d1bf82a6c4a7666b86
I20240502 07:58:43.085799 3629 log_verifier.cc:126] Checking tablet d14545ba1e474960909a1b140b6a315c
I20240502 07:58:43.229323 3629 log_verifier.cc:177] Verified matching terms for 290 ops in tablet d14545ba1e474960909a1b140b6a315c
I20240502 07:58:43.229499 3629 log_verifier.cc:126] Checking tablet d41a2cb7b69e40d9a681384e5ad0a451
I20240502 07:58:43.362444 3629 log_verifier.cc:177] Verified matching terms for 278 ops in tablet d41a2cb7b69e40d9a681384e5ad0a451
I20240502 07:58:43.362641 3629 log_verifier.cc:126] Checking tablet eede21c622d8435ba1efa473ef97b1d1
I20240502 07:58:43.454957 3629 log_verifier.cc:177] Verified matching terms for 166 ops in tablet eede21c622d8435ba1efa473ef97b1d1
I20240502 07:58:43.455111 3629 log_verifier.cc:126] Checking tablet f1a11f8d412949218b8af5de7c63505f
I20240502 07:58:43.589890 3629 log_verifier.cc:177] Verified matching terms for 294 ops in tablet f1a11f8d412949218b8af5de7c63505f
I20240502 07:58:43.591897 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.72:0
--local_ip_for_outbound_sockets=127.3.139.72
--webserver_interface=127.3.139.72
--webserver_port=0
--tserver_master_addrs=127.3.139.126:34225
--builtin_ntp_servers=127.3.139.84:41319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240502 07:58:43.695565 8253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:43.695739 8253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:43.695793 8253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:43.698678 8253 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:58:43.698734 8253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:43.698812 8253 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.72
I20240502 07:58:43.702236 8253 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:41319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.72:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.3.139.72
--webserver_port=0
--tserver_master_addrs=127.3.139.126:34225
--never_fsync=true
--heap_profile_path=/tmp/kudu.8253
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.72
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:43.702950 8253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:43.704003 8253 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:43.711410 8261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:43.711773 8259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:43.712113 8258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:43.712555 8253 server_base.cc:1034] running on GCE node
I20240502 07:58:43.712844 8253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:43.713294 8253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:43.714440 8253 hybrid_clock.cc:648] HybridClock initialized: now 1714636723714426 us; error 24 us; skew 500 ppm
I20240502 07:58:43.716181 8253 webserver.cc:460] Webserver started at http://127.3.139.72:44767/ using document root <none> and password file <none>
I20240502 07:58:43.716652 8253 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:43.716706 8253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:43.716864 8253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:43.718283 8253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/data/instance:
uuid: "389b75652ad34e55b571eb61b5d9c7bc"
format_stamp: "Formatted at 2024-05-02 07:58:43 on dist-test-slave-twcr"
I20240502 07:58:43.718698 8253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/wal/instance:
uuid: "389b75652ad34e55b571eb61b5d9c7bc"
format_stamp: "Formatted at 2024-05-02 07:58:43 on dist-test-slave-twcr"
I20240502 07:58:43.721613 8253 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20240502 07:58:43.723577 8267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:43.724589 8253 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:43.724670 8253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "389b75652ad34e55b571eb61b5d9c7bc"
format_stamp: "Formatted at 2024-05-02 07:58:43 on dist-test-slave-twcr"
I20240502 07:58:43.724733 8253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:43.777597 8253 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:43.778223 8253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:43.778365 8253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:43.778895 8253 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:43.779806 8253 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:43.779851 8253 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:43.779891 8253 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:43.779919 8253 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:43.788914 8253 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.72:36507
I20240502 07:58:43.788947 8380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.72:36507 every 8 connection(s)
I20240502 07:58:43.789739 8253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240502 07:58:43.797506 8381 heartbeater.cc:347] Connected to a master server at 127.3.139.126:34225
I20240502 07:58:43.797691 8381 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:43.797895 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 8253
I20240502 07:58:43.797991 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/wal/instance
I20240502 07:58:43.798166 8381 heartbeater.cc:511] Master 127.3.139.126:34225 requested a full tablet report, sending...
I20240502 07:58:43.799022 5483 ts_manager.cc:194] Registered new tserver with Master: 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72:36507)
I20240502 07:58:43.799546 5483 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.72:39505
I20240502 07:58:43.985917 5872 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:43.986461 7028 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:43.999136 5740 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:44.000097 6132 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:44.002147 8307 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:44.003429 6002 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:58:44.015806 5608 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:44.017897 6263 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:58:44.103148 8307 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:44.110404 7028 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:44.111526 5608 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:44.111647 6263 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:58:44.114300 5872 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:44.119367 6002 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:58:44.126569 5740 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:44.134861 6132 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:44.181813 5892 consensus_queue.cc:237] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 448, Committed index: 448, Last appended: 1.448, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 449 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } }
I20240502 07:58:44.183782 6281 raft_consensus.cc:1261] T bc41794fb8c545149694aa84dde1cdb0 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Refusing update from remote peer 89bf5c0a6b2a4d949851089446f17b96: Log matching property violated. Preceding OpId in replica: term: 1 index: 448. Preceding OpId from leader: term: 1 index: 449. (index mismatch)
I20240502 07:58:44.183879 6018 raft_consensus.cc:1261] T bc41794fb8c545149694aa84dde1cdb0 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Refusing update from remote peer 89bf5c0a6b2a4d949851089446f17b96: Log matching property violated. Preceding OpId in replica: term: 1 index: 448. Preceding OpId from leader: term: 1 index: 449. (index mismatch)
I20240502 07:58:44.184213 8478 consensus_queue.cc:1035] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 449, Last known committed idx: 448, Time since last communication: 0.000s
I20240502 07:58:44.184303 6151 raft_consensus.cc:1261] T bc41794fb8c545149694aa84dde1cdb0 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer 89bf5c0a6b2a4d949851089446f17b96: Log matching property violated. Preceding OpId in replica: term: 1 index: 448. Preceding OpId from leader: term: 1 index: 449. (index mismatch)
I20240502 07:58:44.184438 5757 raft_consensus.cc:1261] T bc41794fb8c545149694aa84dde1cdb0 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer 89bf5c0a6b2a4d949851089446f17b96: Log matching property violated. Preceding OpId in replica: term: 1 index: 448. Preceding OpId from leader: term: 1 index: 449. (index mismatch)
I20240502 07:58:44.184840 8478 consensus_queue.cc:1035] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 449, Last known committed idx: 448, Time since last communication: 0.000s
I20240502 07:58:44.185575 8475 consensus_queue.cc:1035] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 449, Last known committed idx: 448, Time since last communication: 0.000s
I20240502 07:58:44.185883 8476 consensus_queue.cc:1035] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 449, Last known committed idx: 448, Time since last communication: 0.000s
I20240502 07:58:44.187459 8478 raft_consensus.cc:2932] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 LEADER]: Committing config change with OpId 1.449: config changed from index -1 to 449, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 449 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:44.187880 6018 raft_consensus.cc:2932] T bc41794fb8c545149694aa84dde1cdb0 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Committing config change with OpId 1.449: config changed from index -1 to 449, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 449 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:44.189282 5757 raft_consensus.cc:2932] T bc41794fb8c545149694aa84dde1cdb0 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Committing config change with OpId 1.449: config changed from index -1 to 449, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 449 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:44.190605 5486 catalog_manager.cc:5484] T bc41794fb8c545149694aa84dde1cdb0 P d5bc7863cd294f77a932c4a40f98b79d reported cstate change: config changed from index -1 to 449, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New cstate: current_term: 1 leader_uuid: "89bf5c0a6b2a4d949851089446f17b96" committed_config { opid_index: 449 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:44.191450 6281 raft_consensus.cc:2932] T bc41794fb8c545149694aa84dde1cdb0 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Committing config change with OpId 1.449: config changed from index -1 to 449, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 449 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
W20240502 07:58:44.191638 5828 consensus_peers.cc:485] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96 -> Peer 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72:36507): Couldn't send request to peer 389b75652ad34e55b571eb61b5d9c7bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bc41794fb8c545149694aa84dde1cdb0. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:44.193001 6151 raft_consensus.cc:2932] T bc41794fb8c545149694aa84dde1cdb0 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.449: config changed from index -1 to 449, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 449 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:44.199710 6281 consensus_queue.cc:237] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 200, Committed index: 200, Last appended: 1.200, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 201 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } }
I20240502 07:58:44.201268 5892 raft_consensus.cc:1261] T 07d53332bdd34233b3653756770dc9cf P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 200. Preceding OpId from leader: term: 1 index: 201. (index mismatch)
I20240502 07:58:44.201355 5628 raft_consensus.cc:1261] T 07d53332bdd34233b3653756770dc9cf P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 200. Preceding OpId from leader: term: 1 index: 201. (index mismatch)
I20240502 07:58:44.201673 8496 consensus_queue.cc:1035] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 201, Last known committed idx: 200, Time since last communication: 0.000s
I20240502 07:58:44.201794 6151 raft_consensus.cc:1261] T 07d53332bdd34233b3653756770dc9cf P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 200. Preceding OpId from leader: term: 1 index: 201. (index mismatch)
I20240502 07:58:44.202113 8496 consensus_queue.cc:1035] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 201, Last known committed idx: 200, Time since last communication: 0.000s
I20240502 07:58:44.202168 7044 raft_consensus.cc:1261] T 07d53332bdd34233b3653756770dc9cf P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 200. Preceding OpId from leader: term: 1 index: 201. (index mismatch)
I20240502 07:58:44.202747 8496 consensus_queue.cc:1035] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 201, Last known committed idx: 200, Time since last communication: 0.000s
I20240502 07:58:44.202955 8495 consensus_queue.cc:1035] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 201, Last known committed idx: 200, Time since last communication: 0.000s
I20240502 07:58:44.204820 8494 raft_consensus.cc:2932] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 LEADER]: Committing config change with OpId 1.201: config changed from index 102 to 201, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 201 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:44.205271 7044 raft_consensus.cc:2932] T 07d53332bdd34233b3653756770dc9cf P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Committing config change with OpId 1.201: config changed from index 102 to 201, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 201 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
W20240502 07:58:44.205971 6221 consensus_peers.cc:485] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb -> Peer 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72:36507): Couldn't send request to peer 389b75652ad34e55b571eb61b5d9c7bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 07d53332bdd34233b3653756770dc9cf. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:44.206763 5892 raft_consensus.cc:2932] T 07d53332bdd34233b3653756770dc9cf P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Committing config change with OpId 1.201: config changed from index 102 to 201, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 201 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:44.207393 5628 raft_consensus.cc:2932] T 07d53332bdd34233b3653756770dc9cf P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Committing config change with OpId 1.201: config changed from index 102 to 201, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 201 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:44.209007 6151 raft_consensus.cc:2932] T 07d53332bdd34233b3653756770dc9cf P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.201: config changed from index 102 to 201, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 201 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:44.209087 5488 catalog_manager.cc:5484] T 07d53332bdd34233b3653756770dc9cf P 27e3fafad7a24bcda362d2197be87017 reported cstate change: config changed from index 102 to 201, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New cstate: current_term: 1 leader_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" committed_config { opid_index: 201 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:44.216321 5757 consensus_queue.cc:237] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 316, Committed index: 316, Last appended: 1.316, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 317 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } }
I20240502 07:58:44.217998 6018 raft_consensus.cc:1261] T acf2ff61e8ff450ab7fca71fb4191dd6 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Refusing update from remote peer eda55a8566c447cfa749457825b70688: Log matching property violated. Preceding OpId in replica: term: 1 index: 316. Preceding OpId from leader: term: 1 index: 317. (index mismatch)
I20240502 07:58:44.217993 5892 raft_consensus.cc:1261] T acf2ff61e8ff450ab7fca71fb4191dd6 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer eda55a8566c447cfa749457825b70688: Log matching property violated. Preceding OpId in replica: term: 1 index: 316. Preceding OpId from leader: term: 1 index: 317. (index mismatch)
I20240502 07:58:44.218375 8509 consensus_queue.cc:1035] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 317, Last known committed idx: 316, Time since last communication: 0.000s
I20240502 07:58:44.218724 8510 consensus_queue.cc:1035] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 317, Last known committed idx: 316, Time since last communication: 0.000s
I20240502 07:58:44.218796 6151 raft_consensus.cc:1261] T acf2ff61e8ff450ab7fca71fb4191dd6 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer eda55a8566c447cfa749457825b70688: Log matching property violated. Preceding OpId in replica: term: 1 index: 316. Preceding OpId from leader: term: 1 index: 317. (index mismatch)
I20240502 07:58:44.218813 6281 raft_consensus.cc:1261] T acf2ff61e8ff450ab7fca71fb4191dd6 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Refusing update from remote peer eda55a8566c447cfa749457825b70688: Log matching property violated. Preceding OpId in replica: term: 1 index: 316. Preceding OpId from leader: term: 1 index: 317. (index mismatch)
I20240502 07:58:44.219211 8510 consensus_queue.cc:1035] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 317, Last known committed idx: 316, Time since last communication: 0.000s
I20240502 07:58:44.220263 8510 consensus_queue.cc:1035] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 317, Last known committed idx: 316, Time since last communication: 0.000s
I20240502 07:58:44.221266 8509 raft_consensus.cc:2932] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688 [term 1 LEADER]: Committing config change with OpId 1.317: config changed from index -1 to 317, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 317 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:44.221688 6151 raft_consensus.cc:2932] T acf2ff61e8ff450ab7fca71fb4191dd6 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.317: config changed from index -1 to 317, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 317 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
W20240502 07:58:44.222263 5697 consensus_peers.cc:485] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688 -> Peer 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72:36507): Couldn't send request to peer 389b75652ad34e55b571eb61b5d9c7bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: acf2ff61e8ff450ab7fca71fb4191dd6. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:44.222429 6018 raft_consensus.cc:2932] T acf2ff61e8ff450ab7fca71fb4191dd6 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Committing config change with OpId 1.317: config changed from index -1 to 317, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 317 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:44.224238 5482 catalog_manager.cc:5484] T acf2ff61e8ff450ab7fca71fb4191dd6 P e020e7ffe8ce423a8bdefec667161217 reported cstate change: config changed from index -1 to 317, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New cstate: current_term: 1 leader_uuid: "eda55a8566c447cfa749457825b70688" committed_config { opid_index: 317 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:44.224483 6281 raft_consensus.cc:2932] T acf2ff61e8ff450ab7fca71fb4191dd6 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Committing config change with OpId 1.317: config changed from index -1 to 317, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 317 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:44.225271 5892 raft_consensus.cc:2932] T acf2ff61e8ff450ab7fca71fb4191dd6 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Committing config change with OpId 1.317: config changed from index -1 to 317, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 317 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:44.232991 6018 consensus_queue.cc:237] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 290, Committed index: 290, Last appended: 1.290, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 291 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } }
I20240502 07:58:44.234918 7044 raft_consensus.cc:1261] T d14545ba1e474960909a1b140b6a315c P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 290. Preceding OpId from leader: term: 1 index: 291. (index mismatch)
I20240502 07:58:44.234944 5628 raft_consensus.cc:1261] T d14545ba1e474960909a1b140b6a315c P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 290. Preceding OpId from leader: term: 1 index: 291. (index mismatch)
I20240502 07:58:44.235337 6151 raft_consensus.cc:1261] T d14545ba1e474960909a1b140b6a315c P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 290. Preceding OpId from leader: term: 1 index: 291. (index mismatch)
I20240502 07:58:44.235421 8526 consensus_queue.cc:1035] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 291, Last known committed idx: 290, Time since last communication: 0.000s
I20240502 07:58:44.235764 8526 consensus_queue.cc:1035] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 291, Last known committed idx: 290, Time since last communication: 0.000s
I20240502 07:58:44.235960 8163 consensus_queue.cc:1035] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 291, Last known committed idx: 290, Time since last communication: 0.000s
I20240502 07:58:44.237075 5892 raft_consensus.cc:1261] T d14545ba1e474960909a1b140b6a315c P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 290. Preceding OpId from leader: term: 1 index: 291. (index mismatch)
I20240502 07:58:44.237430 8163 consensus_queue.cc:1035] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 291, Last known committed idx: 290, Time since last communication: 0.000s
W20240502 07:58:44.238241 5959 consensus_peers.cc:485] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d -> Peer 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72:36507): Couldn't send request to peer 389b75652ad34e55b571eb61b5d9c7bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d14545ba1e474960909a1b140b6a315c. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:44.238667 8524 raft_consensus.cc:2932] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [term 1 LEADER]: Committing config change with OpId 1.291: config changed from index 138 to 291, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 291 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:44.239315 6151 raft_consensus.cc:2932] T d14545ba1e474960909a1b140b6a315c P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.291: config changed from index 138 to 291, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 291 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:44.239369 7044 raft_consensus.cc:2932] T d14545ba1e474960909a1b140b6a315c P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Committing config change with OpId 1.291: config changed from index 138 to 291, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 291 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:44.240056 5892 raft_consensus.cc:2932] T d14545ba1e474960909a1b140b6a315c P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Committing config change with OpId 1.291: config changed from index 138 to 291, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 291 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:44.241842 5480 catalog_manager.cc:5484] T d14545ba1e474960909a1b140b6a315c P 27e3fafad7a24bcda362d2197be87017 reported cstate change: config changed from index 138 to 291, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New cstate: current_term: 1 leader_uuid: "d5bc7863cd294f77a932c4a40f98b79d" committed_config { opid_index: 291 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:44.242290 5628 raft_consensus.cc:2932] T d14545ba1e474960909a1b140b6a315c P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Committing config change with OpId 1.291: config changed from index 138 to 291, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 291 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:44.249612 6018 consensus_queue.cc:237] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } }
I20240502 07:58:44.250926 5628 raft_consensus.cc:1261] T cd070443095547d1bf82a6c4a7666b86 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:58:44.251015 7044 raft_consensus.cc:1261] T cd070443095547d1bf82a6c4a7666b86 P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:58:44.251036 6151 raft_consensus.cc:1261] T cd070443095547d1bf82a6c4a7666b86 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:58:44.251068 5757 raft_consensus.cc:1261] T cd070443095547d1bf82a6c4a7666b86 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:58:44.251330 8163 consensus_queue.cc:1035] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:58:44.251464 8524 consensus_queue.cc:1035] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:58:44.251597 8525 consensus_queue.cc:1035] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:58:44.251900 8526 consensus_queue.cc:1035] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20240502 07:58:44.252205 5959 consensus_peers.cc:485] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d -> Peer 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72:36507): Couldn't send request to peer 389b75652ad34e55b571eb61b5d9c7bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cd070443095547d1bf82a6c4a7666b86. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:44.253633 8532 raft_consensus.cc:2932] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:44.254019 5757 raft_consensus.cc:2932] T cd070443095547d1bf82a6c4a7666b86 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:44.254820 6151 raft_consensus.cc:2932] T cd070443095547d1bf82a6c4a7666b86 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:44.255537 5628 raft_consensus.cc:2932] T cd070443095547d1bf82a6c4a7666b86 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:44.256403 7044 raft_consensus.cc:2932] T cd070443095547d1bf82a6c4a7666b86 P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:44.256491 5483 catalog_manager.cc:5484] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d reported cstate change: config changed from index 4 to 5, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New cstate: current_term: 1 leader_uuid: "d5bc7863cd294f77a932c4a40f98b79d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:44.615671 8593 ts_tablet_manager.cc:902] T 07d53332bdd34233b3653756770dc9cf P 389b75652ad34e55b571eb61b5d9c7bc: Initiating tablet copy from peer d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70:41047)
I20240502 07:58:44.616353 8593 tablet_copy_client.cc:308] T 07d53332bdd34233b3653756770dc9cf P 389b75652ad34e55b571eb61b5d9c7bc: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.70:41047
I20240502 07:58:44.619482 6303 tablet_copy_service.cc:140] P d58f6d1ff8cf472f8a9edf3700e64bfb: Received BeginTabletCopySession request for tablet 07d53332bdd34233b3653756770dc9cf from peer 389b75652ad34e55b571eb61b5d9c7bc ({username='slave'} at 127.3.139.72:35179)
I20240502 07:58:44.619585 6303 tablet_copy_service.cc:161] P d58f6d1ff8cf472f8a9edf3700e64bfb: Beginning new tablet copy session on tablet 07d53332bdd34233b3653756770dc9cf from peer 389b75652ad34e55b571eb61b5d9c7bc at {username='slave'} at 127.3.139.72:35179: session id = 389b75652ad34e55b571eb61b5d9c7bc-07d53332bdd34233b3653756770dc9cf
I20240502 07:58:44.620329 6303 tablet_copy_source_session.cc:205] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:44.623301 8593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07d53332bdd34233b3653756770dc9cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:44.627875 8593 tablet_copy_client.cc:783] T 07d53332bdd34233b3653756770dc9cf P 389b75652ad34e55b571eb61b5d9c7bc: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:44.628038 8593 tablet_copy_client.cc:647] T 07d53332bdd34233b3653756770dc9cf P 389b75652ad34e55b571eb61b5d9c7bc: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:44.629608 8593 tablet_copy_client.cc:523] T 07d53332bdd34233b3653756770dc9cf P 389b75652ad34e55b571eb61b5d9c7bc: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:44.631024 8593 tablet_bootstrap.cc:492] T 07d53332bdd34233b3653756770dc9cf P 389b75652ad34e55b571eb61b5d9c7bc: Bootstrap starting.
I20240502 07:58:44.646714 8593 log.cc:824] T 07d53332bdd34233b3653756770dc9cf P 389b75652ad34e55b571eb61b5d9c7bc: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:44.677177 8600 ts_tablet_manager.cc:902] T acf2ff61e8ff450ab7fca71fb4191dd6 P 389b75652ad34e55b571eb61b5d9c7bc: Initiating tablet copy from peer eda55a8566c447cfa749457825b70688 (127.3.139.66:39925)
I20240502 07:58:44.677651 8600 tablet_copy_client.cc:308] T acf2ff61e8ff450ab7fca71fb4191dd6 P 389b75652ad34e55b571eb61b5d9c7bc: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.66:39925
I20240502 07:58:44.681394 5781 tablet_copy_service.cc:140] P eda55a8566c447cfa749457825b70688: Received BeginTabletCopySession request for tablet acf2ff61e8ff450ab7fca71fb4191dd6 from peer 389b75652ad34e55b571eb61b5d9c7bc ({username='slave'} at 127.3.139.72:53673)
I20240502 07:58:44.681519 5781 tablet_copy_service.cc:161] P eda55a8566c447cfa749457825b70688: Beginning new tablet copy session on tablet acf2ff61e8ff450ab7fca71fb4191dd6 from peer 389b75652ad34e55b571eb61b5d9c7bc at {username='slave'} at 127.3.139.72:53673: session id = 389b75652ad34e55b571eb61b5d9c7bc-acf2ff61e8ff450ab7fca71fb4191dd6
I20240502 07:58:44.682478 5781 tablet_copy_source_session.cc:205] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:44.683616 8600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acf2ff61e8ff450ab7fca71fb4191dd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:44.685151 8600 tablet_copy_client.cc:783] T acf2ff61e8ff450ab7fca71fb4191dd6 P 389b75652ad34e55b571eb61b5d9c7bc: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:44.685277 8600 tablet_copy_client.cc:647] T acf2ff61e8ff450ab7fca71fb4191dd6 P 389b75652ad34e55b571eb61b5d9c7bc: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:44.686905 8600 tablet_copy_client.cc:523] T acf2ff61e8ff450ab7fca71fb4191dd6 P 389b75652ad34e55b571eb61b5d9c7bc: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:44.688127 8600 tablet_bootstrap.cc:492] T acf2ff61e8ff450ab7fca71fb4191dd6 P 389b75652ad34e55b571eb61b5d9c7bc: Bootstrap starting.
I20240502 07:58:44.769757 8593 tablet_bootstrap.cc:492] T 07d53332bdd34233b3653756770dc9cf P 389b75652ad34e55b571eb61b5d9c7bc: Bootstrap replayed 1/1 log segments. Stats: ops{read=201 overwritten=0 applied=201 ignored=0} inserts{seen=870 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:58:44.770058 8593 tablet_bootstrap.cc:492] T 07d53332bdd34233b3653756770dc9cf P 389b75652ad34e55b571eb61b5d9c7bc: Bootstrap complete.
I20240502 07:58:44.770373 8593 ts_tablet_manager.cc:1367] T 07d53332bdd34233b3653756770dc9cf P 389b75652ad34e55b571eb61b5d9c7bc: Time spent bootstrapping tablet: real 0.139s user 0.125s sys 0.012s
I20240502 07:58:44.772879 8631 ts_tablet_manager.cc:902] T cd070443095547d1bf82a6c4a7666b86 P 389b75652ad34e55b571eb61b5d9c7bc: Initiating tablet copy from peer d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68:44287)
I20240502 07:58:44.772807 8593 raft_consensus.cc:348] T 07d53332bdd34233b3653756770dc9cf P 389b75652ad34e55b571eb61b5d9c7bc [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 201 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } }
I20240502 07:58:44.773082 8593 raft_consensus.cc:729] T 07d53332bdd34233b3653756770dc9cf P 389b75652ad34e55b571eb61b5d9c7bc [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 389b75652ad34e55b571eb61b5d9c7bc, State: Initialized, Role: LEARNER
I20240502 07:58:44.773351 8631 tablet_copy_client.cc:308] T cd070443095547d1bf82a6c4a7666b86 P 389b75652ad34e55b571eb61b5d9c7bc: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.68:44287
I20240502 07:58:44.773444 8593 consensus_queue.cc:260] T 07d53332bdd34233b3653756770dc9cf P 389b75652ad34e55b571eb61b5d9c7bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 201, Last appended: 1.201, Last appended by leader: 201, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 201 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } }
I20240502 07:58:44.774329 8593 ts_tablet_manager.cc:1398] T 07d53332bdd34233b3653756770dc9cf P 389b75652ad34e55b571eb61b5d9c7bc: Time spent starting tablet: real 0.004s user 0.007s sys 0.000s
I20240502 07:58:44.774714 8381 heartbeater.cc:503] Master 127.3.139.126:34225 was elected leader, sending a full tablet report...
I20240502 07:58:44.776778 6303 tablet_copy_service.cc:342] P d58f6d1ff8cf472f8a9edf3700e64bfb: Request end of tablet copy session 389b75652ad34e55b571eb61b5d9c7bc-07d53332bdd34233b3653756770dc9cf received from {username='slave'} at 127.3.139.72:35179
I20240502 07:58:44.776859 6303 tablet_copy_service.cc:433] P d58f6d1ff8cf472f8a9edf3700e64bfb: ending tablet copy session 389b75652ad34e55b571eb61b5d9c7bc-07d53332bdd34233b3653756770dc9cf on tablet 07d53332bdd34233b3653756770dc9cf with peer 389b75652ad34e55b571eb61b5d9c7bc
I20240502 07:58:44.783334 6043 tablet_copy_service.cc:140] P d5bc7863cd294f77a932c4a40f98b79d: Received BeginTabletCopySession request for tablet cd070443095547d1bf82a6c4a7666b86 from peer 389b75652ad34e55b571eb61b5d9c7bc ({username='slave'} at 127.3.139.72:37871)
I20240502 07:58:44.783432 6043 tablet_copy_service.cc:161] P d5bc7863cd294f77a932c4a40f98b79d: Beginning new tablet copy session on tablet cd070443095547d1bf82a6c4a7666b86 from peer 389b75652ad34e55b571eb61b5d9c7bc at {username='slave'} at 127.3.139.72:37871: session id = 389b75652ad34e55b571eb61b5d9c7bc-cd070443095547d1bf82a6c4a7666b86
I20240502 07:58:44.784265 6043 tablet_copy_source_session.cc:205] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:44.785050 8631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd070443095547d1bf82a6c4a7666b86. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:44.786886 8631 tablet_copy_client.cc:783] T cd070443095547d1bf82a6c4a7666b86 P 389b75652ad34e55b571eb61b5d9c7bc: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:44.787017 8631 tablet_copy_client.cc:647] T cd070443095547d1bf82a6c4a7666b86 P 389b75652ad34e55b571eb61b5d9c7bc: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:44.787938 8631 tablet_copy_client.cc:523] T cd070443095547d1bf82a6c4a7666b86 P 389b75652ad34e55b571eb61b5d9c7bc: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:44.788867 8631 tablet_bootstrap.cc:492] T cd070443095547d1bf82a6c4a7666b86 P 389b75652ad34e55b571eb61b5d9c7bc: Bootstrap starting.
I20240502 07:58:44.793216 8631 tablet_bootstrap.cc:492] T cd070443095547d1bf82a6c4a7666b86 P 389b75652ad34e55b571eb61b5d9c7bc: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:58:44.793426 8631 tablet_bootstrap.cc:492] T cd070443095547d1bf82a6c4a7666b86 P 389b75652ad34e55b571eb61b5d9c7bc: Bootstrap complete.
I20240502 07:58:44.793555 8631 ts_tablet_manager.cc:1367] T cd070443095547d1bf82a6c4a7666b86 P 389b75652ad34e55b571eb61b5d9c7bc: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20240502 07:58:44.794082 8631 raft_consensus.cc:348] T cd070443095547d1bf82a6c4a7666b86 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } }
I20240502 07:58:44.794186 8631 raft_consensus.cc:729] T cd070443095547d1bf82a6c4a7666b86 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 389b75652ad34e55b571eb61b5d9c7bc, State: Initialized, Role: LEARNER
I20240502 07:58:44.794273 8631 consensus_queue.cc:260] T cd070443095547d1bf82a6c4a7666b86 P 389b75652ad34e55b571eb61b5d9c7bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } }
I20240502 07:58:44.794504 8631 ts_tablet_manager.cc:1398] T cd070443095547d1bf82a6c4a7666b86 P 389b75652ad34e55b571eb61b5d9c7bc: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240502 07:58:44.796567 8593 ts_tablet_manager.cc:902] T bc41794fb8c545149694aa84dde1cdb0 P 389b75652ad34e55b571eb61b5d9c7bc: Initiating tablet copy from peer 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67:37141)
I20240502 07:58:44.796990 8593 tablet_copy_client.cc:308] T bc41794fb8c545149694aa84dde1cdb0 P 389b75652ad34e55b571eb61b5d9c7bc: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.67:37141
I20240502 07:58:44.798149 6043 tablet_copy_service.cc:342] P d5bc7863cd294f77a932c4a40f98b79d: Request end of tablet copy session 389b75652ad34e55b571eb61b5d9c7bc-cd070443095547d1bf82a6c4a7666b86 received from {username='slave'} at 127.3.139.72:37871
I20240502 07:58:44.798223 6043 tablet_copy_service.cc:433] P d5bc7863cd294f77a932c4a40f98b79d: ending tablet copy session 389b75652ad34e55b571eb61b5d9c7bc-cd070443095547d1bf82a6c4a7666b86 on tablet cd070443095547d1bf82a6c4a7666b86 with peer 389b75652ad34e55b571eb61b5d9c7bc
I20240502 07:58:44.801244 5912 tablet_copy_service.cc:140] P 89bf5c0a6b2a4d949851089446f17b96: Received BeginTabletCopySession request for tablet bc41794fb8c545149694aa84dde1cdb0 from peer 389b75652ad34e55b571eb61b5d9c7bc ({username='slave'} at 127.3.139.72:50453)
I20240502 07:58:44.801374 5912 tablet_copy_service.cc:161] P 89bf5c0a6b2a4d949851089446f17b96: Beginning new tablet copy session on tablet bc41794fb8c545149694aa84dde1cdb0 from peer 389b75652ad34e55b571eb61b5d9c7bc at {username='slave'} at 127.3.139.72:50453: session id = 389b75652ad34e55b571eb61b5d9c7bc-bc41794fb8c545149694aa84dde1cdb0
I20240502 07:58:44.802686 5912 tablet_copy_source_session.cc:205] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:44.803629 8593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc41794fb8c545149694aa84dde1cdb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:44.805023 8593 tablet_copy_client.cc:783] T bc41794fb8c545149694aa84dde1cdb0 P 389b75652ad34e55b571eb61b5d9c7bc: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:44.805143 8593 tablet_copy_client.cc:647] T bc41794fb8c545149694aa84dde1cdb0 P 389b75652ad34e55b571eb61b5d9c7bc: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:44.807428 8593 tablet_copy_client.cc:523] T bc41794fb8c545149694aa84dde1cdb0 P 389b75652ad34e55b571eb61b5d9c7bc: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:44.808509 8593 tablet_bootstrap.cc:492] T bc41794fb8c545149694aa84dde1cdb0 P 389b75652ad34e55b571eb61b5d9c7bc: Bootstrap starting.
I20240502 07:58:44.828190 8631 ts_tablet_manager.cc:902] T d14545ba1e474960909a1b140b6a315c P 389b75652ad34e55b571eb61b5d9c7bc: Initiating tablet copy from peer d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68:44287)
I20240502 07:58:44.828567 8631 tablet_copy_client.cc:308] T d14545ba1e474960909a1b140b6a315c P 389b75652ad34e55b571eb61b5d9c7bc: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.68:44287
I20240502 07:58:44.829030 6043 tablet_copy_service.cc:140] P d5bc7863cd294f77a932c4a40f98b79d: Received BeginTabletCopySession request for tablet d14545ba1e474960909a1b140b6a315c from peer 389b75652ad34e55b571eb61b5d9c7bc ({username='slave'} at 127.3.139.72:37871)
I20240502 07:58:44.829119 6043 tablet_copy_service.cc:161] P d5bc7863cd294f77a932c4a40f98b79d: Beginning new tablet copy session on tablet d14545ba1e474960909a1b140b6a315c from peer 389b75652ad34e55b571eb61b5d9c7bc at {username='slave'} at 127.3.139.72:37871: session id = 389b75652ad34e55b571eb61b5d9c7bc-d14545ba1e474960909a1b140b6a315c
I20240502 07:58:44.830441 6043 tablet_copy_source_session.cc:205] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:44.831204 8631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d14545ba1e474960909a1b140b6a315c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:44.832731 8631 tablet_copy_client.cc:783] T d14545ba1e474960909a1b140b6a315c P 389b75652ad34e55b571eb61b5d9c7bc: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:44.832854 8631 tablet_copy_client.cc:647] T d14545ba1e474960909a1b140b6a315c P 389b75652ad34e55b571eb61b5d9c7bc: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:44.835902 8631 tablet_copy_client.cc:523] T d14545ba1e474960909a1b140b6a315c P 389b75652ad34e55b571eb61b5d9c7bc: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:44.836782 8631 tablet_bootstrap.cc:492] T d14545ba1e474960909a1b140b6a315c P 389b75652ad34e55b571eb61b5d9c7bc: Bootstrap starting.
I20240502 07:58:44.910917 8600 tablet_bootstrap.cc:492] T acf2ff61e8ff450ab7fca71fb4191dd6 P 389b75652ad34e55b571eb61b5d9c7bc: Bootstrap replayed 1/1 log segments. Stats: ops{read=317 overwritten=0 applied=317 ignored=0} inserts{seen=948 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:58:44.911285 8600 tablet_bootstrap.cc:492] T acf2ff61e8ff450ab7fca71fb4191dd6 P 389b75652ad34e55b571eb61b5d9c7bc: Bootstrap complete.
I20240502 07:58:44.911496 8600 ts_tablet_manager.cc:1367] T acf2ff61e8ff450ab7fca71fb4191dd6 P 389b75652ad34e55b571eb61b5d9c7bc: Time spent bootstrapping tablet: real 0.223s user 0.204s sys 0.012s
I20240502 07:58:44.911962 8600 raft_consensus.cc:348] T acf2ff61e8ff450ab7fca71fb4191dd6 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 317 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } }
I20240502 07:58:44.912107 8600 raft_consensus.cc:729] T acf2ff61e8ff450ab7fca71fb4191dd6 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 389b75652ad34e55b571eb61b5d9c7bc, State: Initialized, Role: LEARNER
I20240502 07:58:44.912225 8600 consensus_queue.cc:260] T acf2ff61e8ff450ab7fca71fb4191dd6 P 389b75652ad34e55b571eb61b5d9c7bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 317, Last appended: 1.317, Last appended by leader: 317, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 317 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } }
I20240502 07:58:44.912515 8600 ts_tablet_manager.cc:1398] T acf2ff61e8ff450ab7fca71fb4191dd6 P 389b75652ad34e55b571eb61b5d9c7bc: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:44.913122 5781 tablet_copy_service.cc:342] P eda55a8566c447cfa749457825b70688: Request end of tablet copy session 389b75652ad34e55b571eb61b5d9c7bc-acf2ff61e8ff450ab7fca71fb4191dd6 received from {username='slave'} at 127.3.139.72:53673
I20240502 07:58:44.913228 5781 tablet_copy_service.cc:433] P eda55a8566c447cfa749457825b70688: ending tablet copy session 389b75652ad34e55b571eb61b5d9c7bc-acf2ff61e8ff450ab7fca71fb4191dd6 on tablet acf2ff61e8ff450ab7fca71fb4191dd6 with peer 389b75652ad34e55b571eb61b5d9c7bc
I20240502 07:58:45.041803 8631 tablet_bootstrap.cc:492] T d14545ba1e474960909a1b140b6a315c P 389b75652ad34e55b571eb61b5d9c7bc: Bootstrap replayed 1/1 log segments. Stats: ops{read=291 overwritten=0 applied=291 ignored=0} inserts{seen=973 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:58:45.042070 8631 tablet_bootstrap.cc:492] T d14545ba1e474960909a1b140b6a315c P 389b75652ad34e55b571eb61b5d9c7bc: Bootstrap complete.
I20240502 07:58:45.042258 8631 ts_tablet_manager.cc:1367] T d14545ba1e474960909a1b140b6a315c P 389b75652ad34e55b571eb61b5d9c7bc: Time spent bootstrapping tablet: real 0.206s user 0.182s sys 0.012s
I20240502 07:58:45.042641 8631 raft_consensus.cc:348] T d14545ba1e474960909a1b140b6a315c P 389b75652ad34e55b571eb61b5d9c7bc [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 291 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } }
I20240502 07:58:45.042753 8631 raft_consensus.cc:729] T d14545ba1e474960909a1b140b6a315c P 389b75652ad34e55b571eb61b5d9c7bc [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 389b75652ad34e55b571eb61b5d9c7bc, State: Initialized, Role: LEARNER
I20240502 07:58:45.042833 8631 consensus_queue.cc:260] T d14545ba1e474960909a1b140b6a315c P 389b75652ad34e55b571eb61b5d9c7bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 291, Last appended: 1.291, Last appended by leader: 291, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 291 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } }
I20240502 07:58:45.043125 8631 ts_tablet_manager.cc:1398] T d14545ba1e474960909a1b140b6a315c P 389b75652ad34e55b571eb61b5d9c7bc: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240502 07:58:45.043562 6043 tablet_copy_service.cc:342] P d5bc7863cd294f77a932c4a40f98b79d: Request end of tablet copy session 389b75652ad34e55b571eb61b5d9c7bc-d14545ba1e474960909a1b140b6a315c received from {username='slave'} at 127.3.139.72:37871
I20240502 07:58:45.043645 6043 tablet_copy_service.cc:433] P d5bc7863cd294f77a932c4a40f98b79d: ending tablet copy session 389b75652ad34e55b571eb61b5d9c7bc-d14545ba1e474960909a1b140b6a315c on tablet d14545ba1e474960909a1b140b6a315c with peer 389b75652ad34e55b571eb61b5d9c7bc
I20240502 07:58:45.091787 8593 tablet_bootstrap.cc:492] T bc41794fb8c545149694aa84dde1cdb0 P 389b75652ad34e55b571eb61b5d9c7bc: Bootstrap replayed 1/1 log segments. Stats: ops{read=449 overwritten=0 applied=449 ignored=0} inserts{seen=1080 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:58:45.092025 8593 tablet_bootstrap.cc:492] T bc41794fb8c545149694aa84dde1cdb0 P 389b75652ad34e55b571eb61b5d9c7bc: Bootstrap complete.
I20240502 07:58:45.092206 8593 ts_tablet_manager.cc:1367] T bc41794fb8c545149694aa84dde1cdb0 P 389b75652ad34e55b571eb61b5d9c7bc: Time spent bootstrapping tablet: real 0.284s user 0.246s sys 0.024s
I20240502 07:58:45.092588 8593 raft_consensus.cc:348] T bc41794fb8c545149694aa84dde1cdb0 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 449 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } }
I20240502 07:58:45.092698 8593 raft_consensus.cc:729] T bc41794fb8c545149694aa84dde1cdb0 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 389b75652ad34e55b571eb61b5d9c7bc, State: Initialized, Role: LEARNER
I20240502 07:58:45.092787 8593 consensus_queue.cc:260] T bc41794fb8c545149694aa84dde1cdb0 P 389b75652ad34e55b571eb61b5d9c7bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 449, Last appended: 1.449, Last appended by leader: 449, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 449 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } }
I20240502 07:58:45.093009 8593 ts_tablet_manager.cc:1398] T bc41794fb8c545149694aa84dde1cdb0 P 389b75652ad34e55b571eb61b5d9c7bc: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:45.093614 5912 tablet_copy_service.cc:342] P 89bf5c0a6b2a4d949851089446f17b96: Request end of tablet copy session 389b75652ad34e55b571eb61b5d9c7bc-bc41794fb8c545149694aa84dde1cdb0 received from {username='slave'} at 127.3.139.72:50453
I20240502 07:58:45.093722 5912 tablet_copy_service.cc:433] P 89bf5c0a6b2a4d949851089446f17b96: ending tablet copy session 389b75652ad34e55b571eb61b5d9c7bc-bc41794fb8c545149694aa84dde1cdb0 on tablet bc41794fb8c545149694aa84dde1cdb0 with peer 389b75652ad34e55b571eb61b5d9c7bc
I20240502 07:58:45.169464 8322 raft_consensus.cc:1203] T acf2ff61e8ff450ab7fca71fb4191dd6 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 LEARNER]: Deduplicated request from leader. Original: 1.316->[1.317-1.317] Dedup: 1.317->[]
I20240502 07:58:45.200949 8322 raft_consensus.cc:1203] T 07d53332bdd34233b3653756770dc9cf P 389b75652ad34e55b571eb61b5d9c7bc [term 1 LEARNER]: Deduplicated request from leader. Original: 1.200->[1.201-1.201] Dedup: 1.201->[]
I20240502 07:58:45.211117 8322 raft_consensus.cc:1203] T cd070443095547d1bf82a6c4a7666b86 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240502 07:58:45.280283 8322 raft_consensus.cc:1203] T d14545ba1e474960909a1b140b6a315c P 389b75652ad34e55b571eb61b5d9c7bc [term 1 LEARNER]: Deduplicated request from leader. Original: 1.290->[1.291-1.291] Dedup: 1.291->[]
I20240502 07:58:45.370121 8322 raft_consensus.cc:1203] T bc41794fb8c545149694aa84dde1cdb0 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 LEARNER]: Deduplicated request from leader. Original: 1.448->[1.449-1.449] Dedup: 1.449->[]
I20240502 07:58:45.578140 8690 raft_consensus.cc:1050] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb: attempting to promote NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc to VOTER
I20240502 07:58:45.578598 8690 consensus_queue.cc:237] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 201, Committed index: 201, Last appended: 1.201, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 202 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } }
I20240502 07:58:45.580128 7044 raft_consensus.cc:1261] T 07d53332bdd34233b3653756770dc9cf P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 201. Preceding OpId from leader: term: 1 index: 202. (index mismatch)
I20240502 07:58:45.580214 5891 raft_consensus.cc:1261] T 07d53332bdd34233b3653756770dc9cf P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 201. Preceding OpId from leader: term: 1 index: 202. (index mismatch)
I20240502 07:58:45.580299 5628 raft_consensus.cc:1261] T 07d53332bdd34233b3653756770dc9cf P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 201. Preceding OpId from leader: term: 1 index: 202. (index mismatch)
I20240502 07:58:45.580430 8322 raft_consensus.cc:1261] T 07d53332bdd34233b3653756770dc9cf P 389b75652ad34e55b571eb61b5d9c7bc [term 1 LEARNER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 201. Preceding OpId from leader: term: 1 index: 202. (index mismatch)
I20240502 07:58:45.580667 8494 consensus_queue.cc:1035] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 202, Last known committed idx: 201, Time since last communication: 0.000s
I20240502 07:58:45.580947 6151 raft_consensus.cc:1261] T 07d53332bdd34233b3653756770dc9cf P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 201. Preceding OpId from leader: term: 1 index: 202. (index mismatch)
I20240502 07:58:45.580929 8730 consensus_queue.cc:1035] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 202, Last known committed idx: 201, Time since last communication: 0.000s
I20240502 07:58:45.581204 8732 consensus_queue.cc:1035] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 202, Last known committed idx: 201, Time since last communication: 0.000s
I20240502 07:58:45.581351 8494 consensus_queue.cc:1035] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 202, Last known committed idx: 201, Time since last communication: 0.000s
I20240502 07:58:45.581573 8731 consensus_queue.cc:1035] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 202, Last known committed idx: 201, Time since last communication: 0.000s
I20240502 07:58:45.583333 8732 raft_consensus.cc:2932] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 LEADER]: Committing config change with OpId 1.202: config changed from index 201 to 202, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 202 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.583715 6151 raft_consensus.cc:2932] T 07d53332bdd34233b3653756770dc9cf P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.202: config changed from index 201 to 202, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 202 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.585610 5628 raft_consensus.cc:2932] T 07d53332bdd34233b3653756770dc9cf P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Committing config change with OpId 1.202: config changed from index 201 to 202, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 202 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.586052 7044 raft_consensus.cc:2932] T 07d53332bdd34233b3653756770dc9cf P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Committing config change with OpId 1.202: config changed from index 201 to 202, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 202 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.586302 5480 catalog_manager.cc:5484] T 07d53332bdd34233b3653756770dc9cf P e020e7ffe8ce423a8bdefec667161217 reported cstate change: config changed from index 201 to 202, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" committed_config { opid_index: 202 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.587503 5891 raft_consensus.cc:2932] T 07d53332bdd34233b3653756770dc9cf P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Committing config change with OpId 1.202: config changed from index 201 to 202, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 202 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.588503 8322 raft_consensus.cc:2932] T 07d53332bdd34233b3653756770dc9cf P 389b75652ad34e55b571eb61b5d9c7bc [term 1 FOLLOWER]: Committing config change with OpId 1.202: config changed from index 201 to 202, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 202 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.594362 6281 consensus_queue.cc:237] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 202, Committed index: 202, Last appended: 1.202, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 203 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } }
I20240502 07:58:45.595563 7044 raft_consensus.cc:1261] T 07d53332bdd34233b3653756770dc9cf P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 202. Preceding OpId from leader: term: 1 index: 203. (index mismatch)
I20240502 07:58:45.595620 8322 raft_consensus.cc:1261] T 07d53332bdd34233b3653756770dc9cf P 389b75652ad34e55b571eb61b5d9c7bc [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 202. Preceding OpId from leader: term: 1 index: 203. (index mismatch)
I20240502 07:58:45.595656 5628 raft_consensus.cc:1261] T 07d53332bdd34233b3653756770dc9cf P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 202. Preceding OpId from leader: term: 1 index: 203. (index mismatch)
I20240502 07:58:45.595971 6151 raft_consensus.cc:1261] T 07d53332bdd34233b3653756770dc9cf P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 202. Preceding OpId from leader: term: 1 index: 203. (index mismatch)
I20240502 07:58:45.595983 8732 consensus_queue.cc:1035] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 203, Last known committed idx: 202, Time since last communication: 0.000s
I20240502 07:58:45.596222 8690 consensus_queue.cc:1035] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 203, Last known committed idx: 202, Time since last communication: 0.000s
I20240502 07:58:45.596341 8494 consensus_queue.cc:1035] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 203, Last known committed idx: 202, Time since last communication: 0.000s
I20240502 07:58:45.596474 8673 consensus_queue.cc:1035] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 203, Last known committed idx: 202, Time since last communication: 0.000s
I20240502 07:58:45.597959 8732 raft_consensus.cc:2932] T 07d53332bdd34233b3653756770dc9cf P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 LEADER]: Committing config change with OpId 1.203: config changed from index 202 to 203, VOTER 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67) evicted. New config: { opid_index: 203 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.598371 7044 raft_consensus.cc:2932] T 07d53332bdd34233b3653756770dc9cf P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Committing config change with OpId 1.203: config changed from index 202 to 203, VOTER 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67) evicted. New config: { opid_index: 203 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.598762 8322 raft_consensus.cc:2932] T 07d53332bdd34233b3653756770dc9cf P 389b75652ad34e55b571eb61b5d9c7bc [term 1 FOLLOWER]: Committing config change with OpId 1.203: config changed from index 202 to 203, VOTER 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67) evicted. New config: { opid_index: 203 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.598753 6151 raft_consensus.cc:2932] T 07d53332bdd34233b3653756770dc9cf P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.203: config changed from index 202 to 203, VOTER 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67) evicted. New config: { opid_index: 203 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.600802 5482 catalog_manager.cc:5484] T 07d53332bdd34233b3653756770dc9cf P 27e3fafad7a24bcda362d2197be87017 reported cstate change: config changed from index 202 to 203, VOTER 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67) evicted. New cstate: current_term: 1 leader_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" committed_config { opid_index: 203 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.601379 5474 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 07d53332bdd34233b3653756770dc9cf with cas_config_opid_index 202: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:58:45.601711 5628 raft_consensus.cc:2932] T 07d53332bdd34233b3653756770dc9cf P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Committing config change with OpId 1.203: config changed from index 202 to 203, VOTER 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67) evicted. New config: { opid_index: 203 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.604019 5872 tablet_service.cc:1514] Processing DeleteTablet for tablet 07d53332bdd34233b3653756770dc9cf with delete_type TABLET_DATA_TOMBSTONED (TS 89bf5c0a6b2a4d949851089446f17b96 not found in new config with opid_index 203) from {username='slave'} at 127.0.0.1:42126
I20240502 07:58:45.604652 8744 tablet_replica.cc:330] T 07d53332bdd34233b3653756770dc9cf P 89bf5c0a6b2a4d949851089446f17b96: stopping tablet replica
I20240502 07:58:45.604810 8744 raft_consensus.cc:2229] T 07d53332bdd34233b3653756770dc9cf P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:45.605080 8744 raft_consensus.cc:2258] T 07d53332bdd34233b3653756770dc9cf P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:45.606195 8744 ts_tablet_manager.cc:1875] T 07d53332bdd34233b3653756770dc9cf P 89bf5c0a6b2a4d949851089446f17b96: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:58:45.607823 8744 ts_tablet_manager.cc:1888] T 07d53332bdd34233b3653756770dc9cf P 89bf5c0a6b2a4d949851089446f17b96: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.202
I20240502 07:58:45.607882 8744 log.cc:1196] T 07d53332bdd34233b3653756770dc9cf P 89bf5c0a6b2a4d949851089446f17b96: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal/wals/07d53332bdd34233b3653756770dc9cf
I20240502 07:58:45.608258 5473 catalog_manager.cc:4830] TS 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67:37141): tablet 07d53332bdd34233b3653756770dc9cf (table rebalance_test_table_4 [id=5c1e49d6edf446d5a06063e568da2a0a]) successfully deleted
I20240502 07:58:45.653728 8701 raft_consensus.cc:1050] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d: attempting to promote NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc to VOTER
I20240502 07:58:45.654198 8701 consensus_queue.cc:237] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } }
I20240502 07:58:45.655946 6151 raft_consensus.cc:1261] T cd070443095547d1bf82a6c4a7666b86 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:58:45.655966 7044 raft_consensus.cc:1261] T cd070443095547d1bf82a6c4a7666b86 P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:58:45.656208 5628 raft_consensus.cc:1261] T cd070443095547d1bf82a6c4a7666b86 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:58:45.656297 8763 consensus_queue.cc:1035] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:58:45.656456 8526 consensus_queue.cc:1035] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:58:45.656579 8762 consensus_queue.cc:1035] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:58:45.656634 8322 raft_consensus.cc:1261] T cd070443095547d1bf82a6c4a7666b86 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 LEARNER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:58:45.656725 5760 raft_consensus.cc:1261] T cd070443095547d1bf82a6c4a7666b86 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:58:45.657085 8526 consensus_queue.cc:1035] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:58:45.657428 8526 consensus_queue.cc:1035] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:58:45.659566 8763 raft_consensus.cc:2932] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.659973 7044 raft_consensus.cc:2932] T cd070443095547d1bf82a6c4a7666b86 P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.661514 5628 raft_consensus.cc:2932] T cd070443095547d1bf82a6c4a7666b86 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.662721 5483 catalog_manager.cc:5484] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d reported cstate change: config changed from index 5 to 6, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d5bc7863cd294f77a932c4a40f98b79d" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:45.663522 6151 raft_consensus.cc:2932] T cd070443095547d1bf82a6c4a7666b86 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.664407 5760 raft_consensus.cc:2932] T cd070443095547d1bf82a6c4a7666b86 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.664827 8322 raft_consensus.cc:2932] T cd070443095547d1bf82a6c4a7666b86 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.670894 6018 consensus_queue.cc:237] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } }
I20240502 07:58:45.672226 8322 raft_consensus.cc:1261] T cd070443095547d1bf82a6c4a7666b86 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:58:45.672272 5628 raft_consensus.cc:1261] T cd070443095547d1bf82a6c4a7666b86 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:58:45.672415 7044 raft_consensus.cc:1261] T cd070443095547d1bf82a6c4a7666b86 P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:58:45.672602 8526 consensus_queue.cc:1035] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:58:45.672753 6151 raft_consensus.cc:1261] T cd070443095547d1bf82a6c4a7666b86 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:58:45.672966 8163 consensus_queue.cc:1035] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:58:45.673257 8163 consensus_queue.cc:1035] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:58:45.673424 8701 consensus_queue.cc:1035] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:58:45.674852 8163 raft_consensus.cc:2932] T cd070443095547d1bf82a6c4a7666b86 P d5bc7863cd294f77a932c4a40f98b79d [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER eda55a8566c447cfa749457825b70688 (127.3.139.66) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.675908 6151 raft_consensus.cc:2932] T cd070443095547d1bf82a6c4a7666b86 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER eda55a8566c447cfa749457825b70688 (127.3.139.66) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.676571 7044 raft_consensus.cc:2932] T cd070443095547d1bf82a6c4a7666b86 P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER eda55a8566c447cfa749457825b70688 (127.3.139.66) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.676755 5628 raft_consensus.cc:2932] T cd070443095547d1bf82a6c4a7666b86 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER eda55a8566c447cfa749457825b70688 (127.3.139.66) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.678602 8322 raft_consensus.cc:2932] T cd070443095547d1bf82a6c4a7666b86 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER eda55a8566c447cfa749457825b70688 (127.3.139.66) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.679077 5486 catalog_manager.cc:5484] T cd070443095547d1bf82a6c4a7666b86 P 91e82003a656442189a329b29846a841 reported cstate change: config changed from index 6 to 7, VOTER eda55a8566c447cfa749457825b70688 (127.3.139.66) evicted. New cstate: current_term: 1 leader_uuid: "d5bc7863cd294f77a932c4a40f98b79d" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.679177 5473 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet cd070443095547d1bf82a6c4a7666b86 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:58:45.682705 5740 tablet_service.cc:1514] Processing DeleteTablet for tablet cd070443095547d1bf82a6c4a7666b86 with delete_type TABLET_DATA_TOMBSTONED (TS eda55a8566c447cfa749457825b70688 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:41882
I20240502 07:58:45.683686 8779 tablet_replica.cc:330] T cd070443095547d1bf82a6c4a7666b86 P eda55a8566c447cfa749457825b70688: stopping tablet replica
I20240502 07:58:45.683825 8779 raft_consensus.cc:2229] T cd070443095547d1bf82a6c4a7666b86 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:45.684120 8779 raft_consensus.cc:2258] T cd070443095547d1bf82a6c4a7666b86 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:45.686715 8779 ts_tablet_manager.cc:1875] T cd070443095547d1bf82a6c4a7666b86 P eda55a8566c447cfa749457825b70688: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:58:45.688889 8779 ts_tablet_manager.cc:1888] T cd070443095547d1bf82a6c4a7666b86 P eda55a8566c447cfa749457825b70688: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 07:58:45.689006 8779 log.cc:1196] T cd070443095547d1bf82a6c4a7666b86 P eda55a8566c447cfa749457825b70688: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal/wals/cd070443095547d1bf82a6c4a7666b86
I20240502 07:58:45.689460 5473 catalog_manager.cc:4830] TS eda55a8566c447cfa749457825b70688 (127.3.139.66:39925): tablet cd070443095547d1bf82a6c4a7666b86 (table TestTable [id=ce12803e6d704c758d74d2e1fb8a39b4]) successfully deleted
I20240502 07:58:45.692492 5628 consensus_queue.cc:237] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 166, Committed index: 166, Last appended: 1.166, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 167 OBSOLETE_local: false peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } }
I20240502 07:58:45.694428 5760 raft_consensus.cc:1261] T eede21c622d8435ba1efa473ef97b1d1 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 166. Preceding OpId from leader: term: 1 index: 167. (index mismatch)
I20240502 07:58:45.694530 6151 raft_consensus.cc:1261] T eede21c622d8435ba1efa473ef97b1d1 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 166. Preceding OpId from leader: term: 1 index: 167. (index mismatch)
I20240502 07:58:45.694941 8251 consensus_queue.cc:1035] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 167, Last known committed idx: 166, Time since last communication: 0.000s
I20240502 07:58:45.695179 6018 raft_consensus.cc:1261] T eede21c622d8435ba1efa473ef97b1d1 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 166. Preceding OpId from leader: term: 1 index: 167. (index mismatch)
I20240502 07:58:45.695269 8251 consensus_queue.cc:1035] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 167, Last known committed idx: 166, Time since last communication: 0.000s
I20240502 07:58:45.695596 5891 raft_consensus.cc:1261] T eede21c622d8435ba1efa473ef97b1d1 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 166. Preceding OpId from leader: term: 1 index: 167. (index mismatch)
I20240502 07:58:45.696652 8251 consensus_queue.cc:1035] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 167, Last known committed idx: 166, Time since last communication: 0.000s
I20240502 07:58:45.696952 8251 consensus_queue.cc:1035] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 167, Last known committed idx: 166, Time since last communication: 0.000s
I20240502 07:58:45.697772 8251 raft_consensus.cc:2932] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841 [term 1 LEADER]: Committing config change with OpId 1.167: config changed from index -1 to 167, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 167 OBSOLETE_local: false peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:45.699532 6151 raft_consensus.cc:2932] T eede21c622d8435ba1efa473ef97b1d1 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.167: config changed from index -1 to 167, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 167 OBSOLETE_local: false peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:45.700048 5483 catalog_manager.cc:5484] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841 reported cstate change: config changed from index -1 to 167, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New cstate: current_term: 1 leader_uuid: "91e82003a656442189a329b29846a841" committed_config { opid_index: 167 OBSOLETE_local: false peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:45.700771 5760 raft_consensus.cc:2932] T eede21c622d8435ba1efa473ef97b1d1 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Committing config change with OpId 1.167: config changed from index -1 to 167, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 167 OBSOLETE_local: false peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:45.703119 5891 raft_consensus.cc:2932] T eede21c622d8435ba1efa473ef97b1d1 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Committing config change with OpId 1.167: config changed from index -1 to 167, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 167 OBSOLETE_local: false peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:45.703346 6018 raft_consensus.cc:2932] T eede21c622d8435ba1efa473ef97b1d1 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Committing config change with OpId 1.167: config changed from index -1 to 167, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 167 OBSOLETE_local: false peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
W20240502 07:58:45.711660 5566 consensus_peers.cc:485] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841 -> Peer 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72:36507): Couldn't send request to peer 389b75652ad34e55b571eb61b5d9c7bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: eede21c622d8435ba1efa473ef97b1d1. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:45.736912 5628 consensus_queue.cc:237] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 311, Committed index: 311, Last appended: 1.311, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 312 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false replace: true } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } }
I20240502 07:58:45.738575 7044 raft_consensus.cc:1261] T 18a448dbe2af493abefbc728a8f85186 P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 311. Preceding OpId from leader: term: 1 index: 312. (index mismatch)
I20240502 07:58:45.738929 5891 raft_consensus.cc:1261] T 18a448dbe2af493abefbc728a8f85186 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 311. Preceding OpId from leader: term: 1 index: 312. (index mismatch)
W20240502 07:58:45.739157 5566 consensus_peers.cc:485] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 -> Peer 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72:36507): Couldn't send request to peer 389b75652ad34e55b571eb61b5d9c7bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 18a448dbe2af493abefbc728a8f85186. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:45.739284 6018 raft_consensus.cc:1261] T 18a448dbe2af493abefbc728a8f85186 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 311. Preceding OpId from leader: term: 1 index: 312. (index mismatch)
I20240502 07:58:45.739419 8251 consensus_queue.cc:1035] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 312, Last known committed idx: 311, Time since last communication: 0.000s
I20240502 07:58:45.739635 6151 raft_consensus.cc:1261] T 18a448dbe2af493abefbc728a8f85186 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 311. Preceding OpId from leader: term: 1 index: 312. (index mismatch)
I20240502 07:58:45.739710 8800 consensus_queue.cc:1035] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 312, Last known committed idx: 311, Time since last communication: 0.000s
I20240502 07:58:45.740041 8789 consensus_queue.cc:1035] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 312, Last known committed idx: 311, Time since last communication: 0.000s
I20240502 07:58:45.740223 8800 consensus_queue.cc:1035] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 312, Last known committed idx: 311, Time since last communication: 0.000s
I20240502 07:58:45.741979 8251 raft_consensus.cc:2932] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [term 1 LEADER]: Committing config change with OpId 1.312: config changed from index 106 to 312, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 312 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false replace: true } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:45.742412 7044 raft_consensus.cc:2932] T 18a448dbe2af493abefbc728a8f85186 P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Committing config change with OpId 1.312: config changed from index 106 to 312, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 312 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false replace: true } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:45.742461 6151 raft_consensus.cc:2932] T 18a448dbe2af493abefbc728a8f85186 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.312: config changed from index 106 to 312, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 312 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false replace: true } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:45.743896 6018 raft_consensus.cc:2932] T 18a448dbe2af493abefbc728a8f85186 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Committing config change with OpId 1.312: config changed from index 106 to 312, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 312 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false replace: true } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:45.745088 5486 catalog_manager.cc:5484] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 reported cstate change: config changed from index 106 to 312, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New cstate: current_term: 1 leader_uuid: "91e82003a656442189a329b29846a841" committed_config { opid_index: 312 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:45.745872 5891 raft_consensus.cc:2932] T 18a448dbe2af493abefbc728a8f85186 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Committing config change with OpId 1.312: config changed from index 106 to 312, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 312 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false replace: true } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:45.769153 8701 raft_consensus.cc:1050] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d: attempting to promote NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc to VOTER
I20240502 07:58:45.769596 8701 consensus_queue.cc:237] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 291, Committed index: 291, Last appended: 1.291, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 292 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } }
I20240502 07:58:45.771013 8322 raft_consensus.cc:1261] T d14545ba1e474960909a1b140b6a315c P 389b75652ad34e55b571eb61b5d9c7bc [term 1 LEARNER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 291. Preceding OpId from leader: term: 1 index: 292. (index mismatch)
I20240502 07:58:45.771144 5628 raft_consensus.cc:1261] T d14545ba1e474960909a1b140b6a315c P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 291. Preceding OpId from leader: term: 1 index: 292. (index mismatch)
I20240502 07:58:45.771343 5891 raft_consensus.cc:1261] T d14545ba1e474960909a1b140b6a315c P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 291. Preceding OpId from leader: term: 1 index: 292. (index mismatch)
I20240502 07:58:45.771531 6151 raft_consensus.cc:1261] T d14545ba1e474960909a1b140b6a315c P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 291. Preceding OpId from leader: term: 1 index: 292. (index mismatch)
I20240502 07:58:45.771672 7044 raft_consensus.cc:1261] T d14545ba1e474960909a1b140b6a315c P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 291. Preceding OpId from leader: term: 1 index: 292. (index mismatch)
I20240502 07:58:45.771991 8761 consensus_queue.cc:1035] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 292, Last known committed idx: 291, Time since last communication: 0.000s
I20240502 07:58:45.772172 8763 consensus_queue.cc:1035] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 292, Last known committed idx: 291, Time since last communication: 0.000s
I20240502 07:58:45.772400 8526 consensus_queue.cc:1035] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 292, Last known committed idx: 291, Time since last communication: 0.000s
I20240502 07:58:45.772735 8163 consensus_queue.cc:1035] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 292, Last known committed idx: 291, Time since last communication: 0.000s
I20240502 07:58:45.773427 8763 consensus_queue.cc:1035] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 292, Last known committed idx: 291, Time since last communication: 0.000s
I20240502 07:58:45.775243 8163 raft_consensus.cc:2932] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [term 1 LEADER]: Committing config change with OpId 1.292: config changed from index 291 to 292, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 292 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.775616 5891 raft_consensus.cc:2932] T d14545ba1e474960909a1b140b6a315c P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Committing config change with OpId 1.292: config changed from index 291 to 292, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 292 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.775883 6151 raft_consensus.cc:2932] T d14545ba1e474960909a1b140b6a315c P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.292: config changed from index 291 to 292, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 292 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.775933 5628 raft_consensus.cc:2932] T d14545ba1e474960909a1b140b6a315c P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Committing config change with OpId 1.292: config changed from index 291 to 292, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 292 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.777968 8322 raft_consensus.cc:2932] T d14545ba1e474960909a1b140b6a315c P 389b75652ad34e55b571eb61b5d9c7bc [term 1 FOLLOWER]: Committing config change with OpId 1.292: config changed from index 291 to 292, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 292 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.778247 5486 catalog_manager.cc:5484] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d reported cstate change: config changed from index 291 to 292, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d5bc7863cd294f77a932c4a40f98b79d" committed_config { opid_index: 292 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:58:45.778820 8522 raft_consensus.cc:1050] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688: attempting to promote NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc to VOTER
I20240502 07:58:45.779376 8522 consensus_queue.cc:237] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 317, Committed index: 317, Last appended: 1.317, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 318 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } }
I20240502 07:58:45.780189 7044 raft_consensus.cc:2932] T d14545ba1e474960909a1b140b6a315c P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Committing config change with OpId 1.292: config changed from index 291 to 292, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 292 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.781323 6018 consensus_queue.cc:237] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 292, Committed index: 292, Last appended: 1.292, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 293 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } }
I20240502 07:58:45.781507 6281 raft_consensus.cc:1261] T acf2ff61e8ff450ab7fca71fb4191dd6 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Refusing update from remote peer eda55a8566c447cfa749457825b70688: Log matching property violated. Preceding OpId in replica: term: 1 index: 317. Preceding OpId from leader: term: 1 index: 318. (index mismatch)
I20240502 07:58:45.782152 8522 consensus_queue.cc:1035] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 318, Last known committed idx: 317, Time since last communication: 0.000s
I20240502 07:58:45.783427 5892 raft_consensus.cc:1261] T d14545ba1e474960909a1b140b6a315c P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 292. Preceding OpId from leader: term: 1 index: 293. (index mismatch)
I20240502 07:58:45.783747 8701 consensus_queue.cc:1035] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 293, Last known committed idx: 292, Time since last communication: 0.000s
I20240502 07:58:45.784158 8335 raft_consensus.cc:1261] T d14545ba1e474960909a1b140b6a315c P 389b75652ad34e55b571eb61b5d9c7bc [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 292. Preceding OpId from leader: term: 1 index: 293. (index mismatch)
I20240502 07:58:45.784221 6018 raft_consensus.cc:1261] T acf2ff61e8ff450ab7fca71fb4191dd6 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Refusing update from remote peer eda55a8566c447cfa749457825b70688: Log matching property violated. Preceding OpId in replica: term: 1 index: 317. Preceding OpId from leader: term: 1 index: 318. (index mismatch)
I20240502 07:58:45.784667 8763 consensus_queue.cc:1035] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 293, Last known committed idx: 292, Time since last communication: 0.000s
I20240502 07:58:45.784807 7044 raft_consensus.cc:1261] T d14545ba1e474960909a1b140b6a315c P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 292. Preceding OpId from leader: term: 1 index: 293. (index mismatch)
I20240502 07:58:45.785038 8322 raft_consensus.cc:1261] T acf2ff61e8ff450ab7fca71fb4191dd6 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 LEARNER]: Refusing update from remote peer eda55a8566c447cfa749457825b70688: Log matching property violated. Preceding OpId in replica: term: 1 index: 317. Preceding OpId from leader: term: 1 index: 318. (index mismatch)
I20240502 07:58:45.785308 6148 raft_consensus.cc:1261] T d14545ba1e474960909a1b140b6a315c P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer d5bc7863cd294f77a932c4a40f98b79d: Log matching property violated. Preceding OpId in replica: term: 1 index: 292. Preceding OpId from leader: term: 1 index: 293. (index mismatch)
I20240502 07:58:45.785516 8763 consensus_queue.cc:1035] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 293, Last known committed idx: 292, Time since last communication: 0.000s
I20240502 07:58:45.786002 8763 consensus_queue.cc:1035] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 293, Last known committed idx: 292, Time since last communication: 0.000s
I20240502 07:58:45.786197 5891 raft_consensus.cc:1261] T acf2ff61e8ff450ab7fca71fb4191dd6 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer eda55a8566c447cfa749457825b70688: Log matching property violated. Preceding OpId in replica: term: 1 index: 317. Preceding OpId from leader: term: 1 index: 318. (index mismatch)
I20240502 07:58:45.786738 8522 consensus_queue.cc:1035] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 318, Last known committed idx: 317, Time since last communication: 0.000s
I20240502 07:58:45.787027 8522 consensus_queue.cc:1035] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688 [LEADER]: Connected to new peer: Peer: permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 318, Last known committed idx: 317, Time since last communication: 0.000s
I20240502 07:58:45.788443 8701 raft_consensus.cc:2932] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d [term 1 LEADER]: Committing config change with OpId 1.293: config changed from index 292 to 293, VOTER 91e82003a656442189a329b29846a841 (127.3.139.65) evicted. New config: { opid_index: 293 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.790099 7044 raft_consensus.cc:2932] T d14545ba1e474960909a1b140b6a315c P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Committing config change with OpId 1.293: config changed from index 292 to 293, VOTER 91e82003a656442189a329b29846a841 (127.3.139.65) evicted. New config: { opid_index: 293 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.790684 5486 catalog_manager.cc:5484] T d14545ba1e474960909a1b140b6a315c P d5bc7863cd294f77a932c4a40f98b79d reported cstate change: config changed from index 292 to 293, VOTER 91e82003a656442189a329b29846a841 (127.3.139.65) evicted. New cstate: current_term: 1 leader_uuid: "d5bc7863cd294f77a932c4a40f98b79d" committed_config { opid_index: 293 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:58:45.791231 6151 raft_consensus.cc:1261] T acf2ff61e8ff450ab7fca71fb4191dd6 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer eda55a8566c447cfa749457825b70688: Log matching property violated. Preceding OpId in replica: term: 1 index: 317. Preceding OpId from leader: term: 1 index: 318. (index mismatch)
I20240502 07:58:45.791879 5473 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet d14545ba1e474960909a1b140b6a315c with cas_config_opid_index 292: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:58:45.792470 8322 raft_consensus.cc:2932] T d14545ba1e474960909a1b140b6a315c P 389b75652ad34e55b571eb61b5d9c7bc [term 1 FOLLOWER]: Committing config change with OpId 1.293: config changed from index 292 to 293, VOTER 91e82003a656442189a329b29846a841 (127.3.139.65) evicted. New config: { opid_index: 293 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.792531 5892 raft_consensus.cc:2932] T d14545ba1e474960909a1b140b6a315c P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Committing config change with OpId 1.293: config changed from index 292 to 293, VOTER 91e82003a656442189a329b29846a841 (127.3.139.65) evicted. New config: { opid_index: 293 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.794407 5608 tablet_service.cc:1514] Processing DeleteTablet for tablet d14545ba1e474960909a1b140b6a315c with delete_type TABLET_DATA_TOMBSTONED (TS 91e82003a656442189a329b29846a841 not found in new config with opid_index 293) from {username='slave'} at 127.0.0.1:54448
I20240502 07:58:45.795696 8522 consensus_queue.cc:1035] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 318, Last known committed idx: 317, Time since last communication: 0.007s
I20240502 07:58:45.796725 8522 consensus_queue.cc:1035] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 318, Last known committed idx: 317, Time since last communication: 0.000s
I20240502 07:58:45.797187 8872 tablet_replica.cc:330] T d14545ba1e474960909a1b140b6a315c P 91e82003a656442189a329b29846a841: stopping tablet replica
I20240502 07:58:45.797593 8872 raft_consensus.cc:2229] T d14545ba1e474960909a1b140b6a315c P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:45.798143 8872 raft_consensus.cc:2258] T d14545ba1e474960909a1b140b6a315c P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:45.799144 6148 raft_consensus.cc:2932] T d14545ba1e474960909a1b140b6a315c P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.293: config changed from index 292 to 293, VOTER 91e82003a656442189a329b29846a841 (127.3.139.65) evicted. New config: { opid_index: 293 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.801338 6151 raft_consensus.cc:2932] T acf2ff61e8ff450ab7fca71fb4191dd6 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.318: config changed from index 317 to 318, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 318 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.802829 8868 raft_consensus.cc:2932] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688 [term 1 LEADER]: Committing config change with OpId 1.318: config changed from index 317 to 318, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 318 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.803535 5480 catalog_manager.cc:5484] T acf2ff61e8ff450ab7fca71fb4191dd6 P e020e7ffe8ce423a8bdefec667161217 reported cstate change: config changed from index 317 to 318, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "eda55a8566c447cfa749457825b70688" committed_config { opid_index: 318 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.805301 5892 raft_consensus.cc:2932] T acf2ff61e8ff450ab7fca71fb4191dd6 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Committing config change with OpId 1.318: config changed from index 317 to 318, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 318 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.805370 6018 raft_consensus.cc:2932] T acf2ff61e8ff450ab7fca71fb4191dd6 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Committing config change with OpId 1.318: config changed from index 317 to 318, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 318 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.805442 8322 raft_consensus.cc:2932] T acf2ff61e8ff450ab7fca71fb4191dd6 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 FOLLOWER]: Committing config change with OpId 1.318: config changed from index 317 to 318, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 318 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.802887 8872 ts_tablet_manager.cc:1875] T d14545ba1e474960909a1b140b6a315c P 91e82003a656442189a329b29846a841: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:58:45.809801 5760 consensus_queue.cc:237] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 318, Committed index: 318, Last appended: 1.318, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 319 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } }
I20240502 07:58:45.809803 6281 raft_consensus.cc:2932] T acf2ff61e8ff450ab7fca71fb4191dd6 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Committing config change with OpId 1.318: config changed from index 317 to 318, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 318 OBSOLETE_local: false peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } attrs { replace: true } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.811198 6151 raft_consensus.cc:1261] T acf2ff61e8ff450ab7fca71fb4191dd6 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer eda55a8566c447cfa749457825b70688: Log matching property violated. Preceding OpId in replica: term: 1 index: 318. Preceding OpId from leader: term: 1 index: 319. (index mismatch)
I20240502 07:58:45.811436 5892 raft_consensus.cc:1261] T acf2ff61e8ff450ab7fca71fb4191dd6 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer eda55a8566c447cfa749457825b70688: Log matching property violated. Preceding OpId in replica: term: 1 index: 318. Preceding OpId from leader: term: 1 index: 319. (index mismatch)
I20240502 07:58:45.811520 6018 raft_consensus.cc:1261] T acf2ff61e8ff450ab7fca71fb4191dd6 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Refusing update from remote peer eda55a8566c447cfa749457825b70688: Log matching property violated. Preceding OpId in replica: term: 1 index: 318. Preceding OpId from leader: term: 1 index: 319. (index mismatch)
I20240502 07:58:45.811563 8522 consensus_queue.cc:1035] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 319, Last known committed idx: 318, Time since last communication: 0.000s
I20240502 07:58:45.811975 8522 consensus_queue.cc:1035] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 319, Last known committed idx: 318, Time since last communication: 0.000s
I20240502 07:58:45.812305 8322 raft_consensus.cc:1261] T acf2ff61e8ff450ab7fca71fb4191dd6 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 FOLLOWER]: Refusing update from remote peer eda55a8566c447cfa749457825b70688: Log matching property violated. Preceding OpId in replica: term: 1 index: 318. Preceding OpId from leader: term: 1 index: 319. (index mismatch)
I20240502 07:58:45.812697 8522 consensus_queue.cc:1035] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688 [LEADER]: Connected to new peer: Peer: permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 319, Last known committed idx: 318, Time since last communication: 0.000s
I20240502 07:58:45.813436 8872 ts_tablet_manager.cc:1888] T d14545ba1e474960909a1b140b6a315c P 91e82003a656442189a329b29846a841: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.292
I20240502 07:58:45.813513 8872 log.cc:1196] T d14545ba1e474960909a1b140b6a315c P 91e82003a656442189a329b29846a841: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal/wals/d14545ba1e474960909a1b140b6a315c
I20240502 07:58:45.814107 8868 consensus_queue.cc:1035] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 319, Last known committed idx: 318, Time since last communication: 0.000s
I20240502 07:58:45.814297 5473 catalog_manager.cc:4830] TS 91e82003a656442189a329b29846a841 (127.3.139.65:43429): tablet d14545ba1e474960909a1b140b6a315c (table rebalance_test_table_0 [id=e1e32c7b8b8c4c05b2f82c3195606811]) successfully deleted
I20240502 07:58:45.814508 8865 raft_consensus.cc:2932] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688 [term 1 LEADER]: Committing config change with OpId 1.319: config changed from index 318 to 319, VOTER d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70) evicted. New config: { opid_index: 319 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.814987 6151 raft_consensus.cc:2932] T acf2ff61e8ff450ab7fca71fb4191dd6 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.319: config changed from index 318 to 319, VOTER d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70) evicted. New config: { opid_index: 319 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.816733 5473 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet acf2ff61e8ff450ab7fca71fb4191dd6 with cas_config_opid_index 318: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:58:45.816751 5480 catalog_manager.cc:5484] T acf2ff61e8ff450ab7fca71fb4191dd6 P eda55a8566c447cfa749457825b70688 reported cstate change: config changed from index 318 to 319, VOTER d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70) evicted. New cstate: current_term: 1 leader_uuid: "eda55a8566c447cfa749457825b70688" committed_config { opid_index: 319 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:58:45.817239 5892 raft_consensus.cc:2932] T acf2ff61e8ff450ab7fca71fb4191dd6 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Committing config change with OpId 1.319: config changed from index 318 to 319, VOTER d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70) evicted. New config: { opid_index: 319 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.817512 6018 raft_consensus.cc:2932] T acf2ff61e8ff450ab7fca71fb4191dd6 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Committing config change with OpId 1.319: config changed from index 318 to 319, VOTER d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70) evicted. New config: { opid_index: 319 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.818722 8322 raft_consensus.cc:2932] T acf2ff61e8ff450ab7fca71fb4191dd6 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 FOLLOWER]: Committing config change with OpId 1.319: config changed from index 318 to 319, VOTER d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70) evicted. New config: { opid_index: 319 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.820277 6263 tablet_service.cc:1514] Processing DeleteTablet for tablet acf2ff61e8ff450ab7fca71fb4191dd6 with delete_type TABLET_DATA_TOMBSTONED (TS d58f6d1ff8cf472f8a9edf3700e64bfb not found in new config with opid_index 319) from {username='slave'} at 127.0.0.1:33086
I20240502 07:58:45.820715 8876 tablet_replica.cc:330] T acf2ff61e8ff450ab7fca71fb4191dd6 P d58f6d1ff8cf472f8a9edf3700e64bfb: stopping tablet replica
I20240502 07:58:45.820816 8876 raft_consensus.cc:2229] T acf2ff61e8ff450ab7fca71fb4191dd6 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:45.820955 8876 raft_consensus.cc:2258] T acf2ff61e8ff450ab7fca71fb4191dd6 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:45.822854 8876 ts_tablet_manager.cc:1875] T acf2ff61e8ff450ab7fca71fb4191dd6 P d58f6d1ff8cf472f8a9edf3700e64bfb: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:58:45.824559 8876 ts_tablet_manager.cc:1888] T acf2ff61e8ff450ab7fca71fb4191dd6 P d58f6d1ff8cf472f8a9edf3700e64bfb: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.318
I20240502 07:58:45.824620 8876 log.cc:1196] T acf2ff61e8ff450ab7fca71fb4191dd6 P d58f6d1ff8cf472f8a9edf3700e64bfb: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/wal/wals/acf2ff61e8ff450ab7fca71fb4191dd6
I20240502 07:58:45.825080 5474 catalog_manager.cc:4830] TS d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70:41047): tablet acf2ff61e8ff450ab7fca71fb4191dd6 (table rebalance_test_table_1 [id=6f5d44f6fbe544099b33ad94e35cee6e]) successfully deleted
I20240502 07:58:45.852567 8703 raft_consensus.cc:1050] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96: attempting to promote NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc to VOTER
I20240502 07:58:45.853111 8703 consensus_queue.cc:237] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 449, Committed index: 449, Last appended: 1.449, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 450 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } }
I20240502 07:58:45.854848 5760 raft_consensus.cc:1261] T bc41794fb8c545149694aa84dde1cdb0 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer 89bf5c0a6b2a4d949851089446f17b96: Log matching property violated. Preceding OpId in replica: term: 1 index: 449. Preceding OpId from leader: term: 1 index: 450. (index mismatch)
I20240502 07:58:45.855257 8477 consensus_queue.cc:1035] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 450, Last known committed idx: 449, Time since last communication: 0.000s
I20240502 07:58:45.855345 8322 raft_consensus.cc:1261] T bc41794fb8c545149694aa84dde1cdb0 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 LEARNER]: Refusing update from remote peer 89bf5c0a6b2a4d949851089446f17b96: Log matching property violated. Preceding OpId in replica: term: 1 index: 449. Preceding OpId from leader: term: 1 index: 450. (index mismatch)
I20240502 07:58:45.855348 6281 raft_consensus.cc:1261] T bc41794fb8c545149694aa84dde1cdb0 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Refusing update from remote peer 89bf5c0a6b2a4d949851089446f17b96: Log matching property violated. Preceding OpId in replica: term: 1 index: 449. Preceding OpId from leader: term: 1 index: 450. (index mismatch)
I20240502 07:58:45.855351 6018 raft_consensus.cc:1261] T bc41794fb8c545149694aa84dde1cdb0 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Refusing update from remote peer 89bf5c0a6b2a4d949851089446f17b96: Log matching property violated. Preceding OpId in replica: term: 1 index: 449. Preceding OpId from leader: term: 1 index: 450. (index mismatch)
I20240502 07:58:45.855667 8477 consensus_queue.cc:1035] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 450, Last known committed idx: 449, Time since last communication: 0.000s
I20240502 07:58:45.855732 6151 raft_consensus.cc:1261] T bc41794fb8c545149694aa84dde1cdb0 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer 89bf5c0a6b2a4d949851089446f17b96: Log matching property violated. Preceding OpId in replica: term: 1 index: 449. Preceding OpId from leader: term: 1 index: 450. (index mismatch)
I20240502 07:58:45.855818 8878 consensus_queue.cc:1035] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 450, Last known committed idx: 449, Time since last communication: 0.000s
I20240502 07:58:45.856029 8703 consensus_queue.cc:1035] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 450, Last known committed idx: 449, Time since last communication: 0.000s
I20240502 07:58:45.856319 8702 consensus_queue.cc:1035] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 450, Last known committed idx: 449, Time since last communication: 0.000s
I20240502 07:58:45.859072 8878 raft_consensus.cc:2932] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 LEADER]: Committing config change with OpId 1.450: config changed from index 449 to 450, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 450 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.859944 8322 raft_consensus.cc:2932] T bc41794fb8c545149694aa84dde1cdb0 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 FOLLOWER]: Committing config change with OpId 1.450: config changed from index 449 to 450, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 450 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.860775 6151 raft_consensus.cc:2932] T bc41794fb8c545149694aa84dde1cdb0 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.450: config changed from index 449 to 450, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 450 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.861224 6018 raft_consensus.cc:2932] T bc41794fb8c545149694aa84dde1cdb0 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Committing config change with OpId 1.450: config changed from index 449 to 450, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 450 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.862924 5486 catalog_manager.cc:5484] T bc41794fb8c545149694aa84dde1cdb0 P 389b75652ad34e55b571eb61b5d9c7bc reported cstate change: config changed from index 449 to 450, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "89bf5c0a6b2a4d949851089446f17b96" committed_config { opid_index: 450 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.864189 5760 raft_consensus.cc:2932] T bc41794fb8c545149694aa84dde1cdb0 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Committing config change with OpId 1.450: config changed from index 449 to 450, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 450 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.865150 6281 raft_consensus.cc:2932] T bc41794fb8c545149694aa84dde1cdb0 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Committing config change with OpId 1.450: config changed from index 449 to 450, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 450 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.869539 5892 consensus_queue.cc:237] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 450, Committed index: 450, Last appended: 1.450, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 451 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } }
I20240502 07:58:45.870713 6018 raft_consensus.cc:1261] T bc41794fb8c545149694aa84dde1cdb0 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Refusing update from remote peer 89bf5c0a6b2a4d949851089446f17b96: Log matching property violated. Preceding OpId in replica: term: 1 index: 450. Preceding OpId from leader: term: 1 index: 451. (index mismatch)
I20240502 07:58:45.870710 8322 raft_consensus.cc:1261] T bc41794fb8c545149694aa84dde1cdb0 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 FOLLOWER]: Refusing update from remote peer 89bf5c0a6b2a4d949851089446f17b96: Log matching property violated. Preceding OpId in replica: term: 1 index: 450. Preceding OpId from leader: term: 1 index: 451. (index mismatch)
I20240502 07:58:45.870898 5760 raft_consensus.cc:1261] T bc41794fb8c545149694aa84dde1cdb0 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer 89bf5c0a6b2a4d949851089446f17b96: Log matching property violated. Preceding OpId in replica: term: 1 index: 450. Preceding OpId from leader: term: 1 index: 451. (index mismatch)
I20240502 07:58:45.871006 6281 raft_consensus.cc:1261] T bc41794fb8c545149694aa84dde1cdb0 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Refusing update from remote peer 89bf5c0a6b2a4d949851089446f17b96: Log matching property violated. Preceding OpId in replica: term: 1 index: 450. Preceding OpId from leader: term: 1 index: 451. (index mismatch)
I20240502 07:58:45.871076 8703 consensus_queue.cc:1035] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 451, Last known committed idx: 450, Time since last communication: 0.000s
I20240502 07:58:45.871358 8880 consensus_queue.cc:1035] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 451, Last known committed idx: 450, Time since last communication: 0.000s
I20240502 07:58:45.871541 8703 consensus_queue.cc:1035] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 451, Last known committed idx: 450, Time since last communication: 0.000s
I20240502 07:58:45.871675 8879 consensus_queue.cc:1035] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 451, Last known committed idx: 450, Time since last communication: 0.000s
I20240502 07:58:45.873355 8879 raft_consensus.cc:2932] T bc41794fb8c545149694aa84dde1cdb0 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 LEADER]: Committing config change with OpId 1.451: config changed from index 450 to 451, VOTER e020e7ffe8ce423a8bdefec667161217 (127.3.139.69) evicted. New config: { opid_index: 451 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.873756 6018 raft_consensus.cc:2932] T bc41794fb8c545149694aa84dde1cdb0 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Committing config change with OpId 1.451: config changed from index 450 to 451, VOTER e020e7ffe8ce423a8bdefec667161217 (127.3.139.69) evicted. New config: { opid_index: 451 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.873768 5760 raft_consensus.cc:2932] T bc41794fb8c545149694aa84dde1cdb0 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Committing config change with OpId 1.451: config changed from index 450 to 451, VOTER e020e7ffe8ce423a8bdefec667161217 (127.3.139.69) evicted. New config: { opid_index: 451 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.874154 8322 raft_consensus.cc:2932] T bc41794fb8c545149694aa84dde1cdb0 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 FOLLOWER]: Committing config change with OpId 1.451: config changed from index 450 to 451, VOTER e020e7ffe8ce423a8bdefec667161217 (127.3.139.69) evicted. New config: { opid_index: 451 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.875214 5473 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet bc41794fb8c545149694aa84dde1cdb0 with cas_config_opid_index 450: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:58:45.875794 6281 raft_consensus.cc:2932] T bc41794fb8c545149694aa84dde1cdb0 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Committing config change with OpId 1.451: config changed from index 450 to 451, VOTER e020e7ffe8ce423a8bdefec667161217 (127.3.139.69) evicted. New config: { opid_index: 451 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.876020 5486 catalog_manager.cc:5484] T bc41794fb8c545149694aa84dde1cdb0 P d5bc7863cd294f77a932c4a40f98b79d reported cstate change: config changed from index 450 to 451, VOTER e020e7ffe8ce423a8bdefec667161217 (127.3.139.69) evicted. New cstate: current_term: 1 leader_uuid: "89bf5c0a6b2a4d949851089446f17b96" committed_config { opid_index: 451 OBSOLETE_local: false peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:45.878656 6132 tablet_service.cc:1514] Processing DeleteTablet for tablet bc41794fb8c545149694aa84dde1cdb0 with delete_type TABLET_DATA_TOMBSTONED (TS e020e7ffe8ce423a8bdefec667161217 not found in new config with opid_index 451) from {username='slave'} at 127.0.0.1:46442
I20240502 07:58:45.879300 8886 tablet_replica.cc:330] T bc41794fb8c545149694aa84dde1cdb0 P e020e7ffe8ce423a8bdefec667161217: stopping tablet replica
I20240502 07:58:45.879485 8886 raft_consensus.cc:2229] T bc41794fb8c545149694aa84dde1cdb0 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:45.879640 8886 raft_consensus.cc:2258] T bc41794fb8c545149694aa84dde1cdb0 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:45.880692 8886 ts_tablet_manager.cc:1875] T bc41794fb8c545149694aa84dde1cdb0 P e020e7ffe8ce423a8bdefec667161217: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:58:45.882292 8886 ts_tablet_manager.cc:1888] T bc41794fb8c545149694aa84dde1cdb0 P e020e7ffe8ce423a8bdefec667161217: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.450
I20240502 07:58:45.882351 8886 log.cc:1196] T bc41794fb8c545149694aa84dde1cdb0 P e020e7ffe8ce423a8bdefec667161217: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/wal/wals/bc41794fb8c545149694aa84dde1cdb0
I20240502 07:58:45.882798 5474 catalog_manager.cc:4830] TS e020e7ffe8ce423a8bdefec667161217 (127.3.139.69:39399): tablet bc41794fb8c545149694aa84dde1cdb0 (table rebalance_test_table_3 [id=d0a0bf64867c4915ad797ab2bcd7101f]) successfully deleted
I20240502 07:58:46.006633 5760 consensus_queue.cc:237] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 292, Committed index: 292, Last appended: 1.292, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 293 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } }
I20240502 07:58:46.008335 6151 raft_consensus.cc:1261] T 635b0a1a0fd64eeeac70025014c9ab81 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer eda55a8566c447cfa749457825b70688: Log matching property violated. Preceding OpId in replica: term: 1 index: 292. Preceding OpId from leader: term: 1 index: 293. (index mismatch)
I20240502 07:58:46.008339 6281 raft_consensus.cc:1261] T 635b0a1a0fd64eeeac70025014c9ab81 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Refusing update from remote peer eda55a8566c447cfa749457825b70688: Log matching property violated. Preceding OpId in replica: term: 1 index: 292. Preceding OpId from leader: term: 1 index: 293. (index mismatch)
I20240502 07:58:46.008646 5628 raft_consensus.cc:1261] T 635b0a1a0fd64eeeac70025014c9ab81 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer eda55a8566c447cfa749457825b70688: Log matching property violated. Preceding OpId in replica: term: 1 index: 292. Preceding OpId from leader: term: 1 index: 293. (index mismatch)
I20240502 07:58:46.008667 5892 raft_consensus.cc:1261] T 635b0a1a0fd64eeeac70025014c9ab81 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer eda55a8566c447cfa749457825b70688: Log matching property violated. Preceding OpId in replica: term: 1 index: 292. Preceding OpId from leader: term: 1 index: 293. (index mismatch)
W20240502 07:58:46.008951 5697 consensus_peers.cc:485] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688 -> Peer 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72:36507): Couldn't send request to peer 389b75652ad34e55b571eb61b5d9c7bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 635b0a1a0fd64eeeac70025014c9ab81. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:46.009119 8867 consensus_queue.cc:1035] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 293, Last known committed idx: 292, Time since last communication: 0.000s
I20240502 07:58:46.009284 8865 consensus_queue.cc:1035] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 293, Last known committed idx: 292, Time since last communication: 0.000s
I20240502 07:58:46.009428 8522 consensus_queue.cc:1035] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 293, Last known committed idx: 292, Time since last communication: 0.000s
I20240502 07:58:46.009616 8875 consensus_queue.cc:1035] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 293, Last known committed idx: 292, Time since last communication: 0.000s
I20240502 07:58:46.011503 8865 raft_consensus.cc:2932] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688 [term 1 LEADER]: Committing config change with OpId 1.293: config changed from index -1 to 293, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 293 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:46.011881 5628 raft_consensus.cc:2932] T 635b0a1a0fd64eeeac70025014c9ab81 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Committing config change with OpId 1.293: config changed from index -1 to 293, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 293 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:46.012187 6281 raft_consensus.cc:2932] T 635b0a1a0fd64eeeac70025014c9ab81 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Committing config change with OpId 1.293: config changed from index -1 to 293, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 293 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:46.012329 5892 raft_consensus.cc:2932] T 635b0a1a0fd64eeeac70025014c9ab81 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Committing config change with OpId 1.293: config changed from index -1 to 293, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 293 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:46.014122 6151 raft_consensus.cc:2932] T 635b0a1a0fd64eeeac70025014c9ab81 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.293: config changed from index -1 to 293, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 293 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:46.014694 5488 catalog_manager.cc:5484] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688 reported cstate change: config changed from index -1 to 293, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New cstate: current_term: 1 leader_uuid: "eda55a8566c447cfa749457825b70688" committed_config { opid_index: 293 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:46.023353 6281 consensus_queue.cc:237] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 278, Committed index: 278, Last appended: 1.278, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 279 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } }
I20240502 07:58:46.024762 6151 raft_consensus.cc:1261] T d41a2cb7b69e40d9a681384e5ad0a451 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 278. Preceding OpId from leader: term: 1 index: 279. (index mismatch)
I20240502 07:58:46.024852 5760 raft_consensus.cc:1261] T d41a2cb7b69e40d9a681384e5ad0a451 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 278. Preceding OpId from leader: term: 1 index: 279. (index mismatch)
I20240502 07:58:46.025157 5628 raft_consensus.cc:1261] T d41a2cb7b69e40d9a681384e5ad0a451 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 278. Preceding OpId from leader: term: 1 index: 279. (index mismatch)
I20240502 07:58:46.025286 7044 raft_consensus.cc:1261] T d41a2cb7b69e40d9a681384e5ad0a451 P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 278. Preceding OpId from leader: term: 1 index: 279. (index mismatch)
I20240502 07:58:46.025264 8730 consensus_queue.cc:1035] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 279, Last known committed idx: 278, Time since last communication: 0.000s
W20240502 07:58:46.025523 6221 consensus_peers.cc:485] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb -> Peer 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72:36507): Couldn't send request to peer 389b75652ad34e55b571eb61b5d9c7bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d41a2cb7b69e40d9a681384e5ad0a451. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:46.025595 8730 consensus_queue.cc:1035] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 279, Last known committed idx: 278, Time since last communication: 0.000s
I20240502 07:58:46.025727 8732 consensus_queue.cc:1035] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 279, Last known committed idx: 278, Time since last communication: 0.000s
I20240502 07:58:46.025935 8731 consensus_queue.cc:1035] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 279, Last known committed idx: 278, Time since last communication: 0.000s
I20240502 07:58:46.027688 8731 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 LEADER]: Committing config change with OpId 1.279: config changed from index 182 to 279, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 279 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:46.028136 5760 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Committing config change with OpId 1.279: config changed from index 182 to 279, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 279 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:46.028281 5628 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Committing config change with OpId 1.279: config changed from index 182 to 279, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 279 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:46.030571 7044 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Committing config change with OpId 1.279: config changed from index 182 to 279, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 279 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:46.030689 5483 catalog_manager.cc:5484] T d41a2cb7b69e40d9a681384e5ad0a451 P 91e82003a656442189a329b29846a841 reported cstate change: config changed from index 182 to 279, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New cstate: current_term: 1 leader_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" committed_config { opid_index: 279 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:46.032097 6151 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.279: config changed from index 182 to 279, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 279 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:46.038666 5628 consensus_queue.cc:237] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 500, Committed index: 500, Last appended: 1.500, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 501 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } }
I20240502 07:58:46.040033 7044 raft_consensus.cc:1261] T 96dcb06388374514951f24393f288eca P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 500. Preceding OpId from leader: term: 1 index: 501. (index mismatch)
I20240502 07:58:46.040256 5892 raft_consensus.cc:1261] T 96dcb06388374514951f24393f288eca P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 500. Preceding OpId from leader: term: 1 index: 501. (index mismatch)
I20240502 07:58:46.040438 6151 raft_consensus.cc:1261] T 96dcb06388374514951f24393f288eca P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 500. Preceding OpId from leader: term: 1 index: 501. (index mismatch)
I20240502 07:58:46.040462 8251 consensus_queue.cc:1035] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 501, Last known committed idx: 500, Time since last communication: 0.000s
I20240502 07:58:46.040634 8800 consensus_queue.cc:1035] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 501, Last known committed idx: 500, Time since last communication: 0.000s
W20240502 07:58:46.040835 5566 consensus_peers.cc:485] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 -> Peer 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72:36507): Couldn't send request to peer 389b75652ad34e55b571eb61b5d9c7bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 96dcb06388374514951f24393f288eca. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:46.040871 8788 consensus_queue.cc:1035] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 501, Last known committed idx: 500, Time since last communication: 0.000s
I20240502 07:58:46.042382 5760 raft_consensus.cc:1261] T 96dcb06388374514951f24393f288eca P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 500. Preceding OpId from leader: term: 1 index: 501. (index mismatch)
I20240502 07:58:46.042722 8788 consensus_queue.cc:1035] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 501, Last known committed idx: 500, Time since last communication: 0.000s
I20240502 07:58:46.043020 8789 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [term 1 LEADER]: Committing config change with OpId 1.501: config changed from index 343 to 501, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 501 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:46.043592 7044 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Committing config change with OpId 1.501: config changed from index 343 to 501, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 501 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:46.043978 5760 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Committing config change with OpId 1.501: config changed from index 343 to 501, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 501 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:46.045001 5892 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Committing config change with OpId 1.501: config changed from index 343 to 501, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 501 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:46.046113 5486 catalog_manager.cc:5484] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 reported cstate change: config changed from index 343 to 501, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New cstate: current_term: 1 leader_uuid: "91e82003a656442189a329b29846a841" committed_config { opid_index: 501 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:46.046970 6151 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.501: config changed from index 343 to 501, NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) added. New config: { opid_index: 501 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } } }
I20240502 07:58:46.091949 8967 ts_tablet_manager.cc:902] T eede21c622d8435ba1efa473ef97b1d1 P 389b75652ad34e55b571eb61b5d9c7bc: Initiating tablet copy from peer 91e82003a656442189a329b29846a841 (127.3.139.65:43429)
I20240502 07:58:46.092504 8967 tablet_copy_client.cc:308] T eede21c622d8435ba1efa473ef97b1d1 P 389b75652ad34e55b571eb61b5d9c7bc: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.65:43429
I20240502 07:58:46.095710 5650 tablet_copy_service.cc:140] P 91e82003a656442189a329b29846a841: Received BeginTabletCopySession request for tablet eede21c622d8435ba1efa473ef97b1d1 from peer 389b75652ad34e55b571eb61b5d9c7bc ({username='slave'} at 127.3.139.72:41667)
I20240502 07:58:46.095799 5650 tablet_copy_service.cc:161] P 91e82003a656442189a329b29846a841: Beginning new tablet copy session on tablet eede21c622d8435ba1efa473ef97b1d1 from peer 389b75652ad34e55b571eb61b5d9c7bc at {username='slave'} at 127.3.139.72:41667: session id = 389b75652ad34e55b571eb61b5d9c7bc-eede21c622d8435ba1efa473ef97b1d1
I20240502 07:58:46.096493 5650 tablet_copy_source_session.cc:205] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:46.097244 8967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eede21c622d8435ba1efa473ef97b1d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:46.098807 8967 tablet_copy_client.cc:783] T eede21c622d8435ba1efa473ef97b1d1 P 389b75652ad34e55b571eb61b5d9c7bc: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:46.098937 8967 tablet_copy_client.cc:647] T eede21c622d8435ba1efa473ef97b1d1 P 389b75652ad34e55b571eb61b5d9c7bc: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:46.100119 8967 tablet_copy_client.cc:523] T eede21c622d8435ba1efa473ef97b1d1 P 389b75652ad34e55b571eb61b5d9c7bc: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:46.101007 8967 tablet_bootstrap.cc:492] T eede21c622d8435ba1efa473ef97b1d1 P 389b75652ad34e55b571eb61b5d9c7bc: Bootstrap starting.
I20240502 07:58:46.171236 8974 ts_tablet_manager.cc:902] T 18a448dbe2af493abefbc728a8f85186 P 389b75652ad34e55b571eb61b5d9c7bc: Initiating tablet copy from peer 91e82003a656442189a329b29846a841 (127.3.139.65:43429)
I20240502 07:58:46.171715 8974 tablet_copy_client.cc:308] T 18a448dbe2af493abefbc728a8f85186 P 389b75652ad34e55b571eb61b5d9c7bc: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.65:43429
I20240502 07:58:46.172180 5650 tablet_copy_service.cc:140] P 91e82003a656442189a329b29846a841: Received BeginTabletCopySession request for tablet 18a448dbe2af493abefbc728a8f85186 from peer 389b75652ad34e55b571eb61b5d9c7bc ({username='slave'} at 127.3.139.72:41667)
I20240502 07:58:46.172298 5650 tablet_copy_service.cc:161] P 91e82003a656442189a329b29846a841: Beginning new tablet copy session on tablet 18a448dbe2af493abefbc728a8f85186 from peer 389b75652ad34e55b571eb61b5d9c7bc at {username='slave'} at 127.3.139.72:41667: session id = 389b75652ad34e55b571eb61b5d9c7bc-18a448dbe2af493abefbc728a8f85186
I20240502 07:58:46.173336 5650 tablet_copy_source_session.cc:205] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:46.174171 8974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18a448dbe2af493abefbc728a8f85186. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:46.176270 8974 tablet_copy_client.cc:783] T 18a448dbe2af493abefbc728a8f85186 P 389b75652ad34e55b571eb61b5d9c7bc: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:46.176482 8974 tablet_copy_client.cc:647] T 18a448dbe2af493abefbc728a8f85186 P 389b75652ad34e55b571eb61b5d9c7bc: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:46.178170 8974 tablet_copy_client.cc:523] T 18a448dbe2af493abefbc728a8f85186 P 389b75652ad34e55b571eb61b5d9c7bc: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:46.179454 8974 tablet_bootstrap.cc:492] T 18a448dbe2af493abefbc728a8f85186 P 389b75652ad34e55b571eb61b5d9c7bc: Bootstrap starting.
I20240502 07:58:46.212697 8967 tablet_bootstrap.cc:492] T eede21c622d8435ba1efa473ef97b1d1 P 389b75652ad34e55b571eb61b5d9c7bc: Bootstrap replayed 1/1 log segments. Stats: ops{read=167 overwritten=0 applied=167 ignored=0} inserts{seen=852 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:58:46.212960 8967 tablet_bootstrap.cc:492] T eede21c622d8435ba1efa473ef97b1d1 P 389b75652ad34e55b571eb61b5d9c7bc: Bootstrap complete.
I20240502 07:58:46.213142 8967 ts_tablet_manager.cc:1367] T eede21c622d8435ba1efa473ef97b1d1 P 389b75652ad34e55b571eb61b5d9c7bc: Time spent bootstrapping tablet: real 0.112s user 0.098s sys 0.012s
I20240502 07:58:46.213536 8967 raft_consensus.cc:348] T eede21c622d8435ba1efa473ef97b1d1 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 167 OBSOLETE_local: false peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } }
I20240502 07:58:46.213663 8967 raft_consensus.cc:729] T eede21c622d8435ba1efa473ef97b1d1 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 389b75652ad34e55b571eb61b5d9c7bc, State: Initialized, Role: LEARNER
I20240502 07:58:46.213761 8967 consensus_queue.cc:260] T eede21c622d8435ba1efa473ef97b1d1 P 389b75652ad34e55b571eb61b5d9c7bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 167, Last appended: 1.167, Last appended by leader: 167, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 167 OBSOLETE_local: false peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } }
I20240502 07:58:46.214020 8967 ts_tablet_manager.cc:1398] T eede21c622d8435ba1efa473ef97b1d1 P 389b75652ad34e55b571eb61b5d9c7bc: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:46.214586 5650 tablet_copy_service.cc:342] P 91e82003a656442189a329b29846a841: Request end of tablet copy session 389b75652ad34e55b571eb61b5d9c7bc-eede21c622d8435ba1efa473ef97b1d1 received from {username='slave'} at 127.3.139.72:41667
I20240502 07:58:46.214675 5650 tablet_copy_service.cc:433] P 91e82003a656442189a329b29846a841: ending tablet copy session 389b75652ad34e55b571eb61b5d9c7bc-eede21c622d8435ba1efa473ef97b1d1 on tablet eede21c622d8435ba1efa473ef97b1d1 with peer 389b75652ad34e55b571eb61b5d9c7bc
I20240502 07:58:46.374795 8974 tablet_bootstrap.cc:492] T 18a448dbe2af493abefbc728a8f85186 P 389b75652ad34e55b571eb61b5d9c7bc: Bootstrap replayed 1/1 log segments. Stats: ops{read=312 overwritten=0 applied=312 ignored=0} inserts{seen=994 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:58:46.375100 8974 tablet_bootstrap.cc:492] T 18a448dbe2af493abefbc728a8f85186 P 389b75652ad34e55b571eb61b5d9c7bc: Bootstrap complete.
I20240502 07:58:46.375291 8974 ts_tablet_manager.cc:1367] T 18a448dbe2af493abefbc728a8f85186 P 389b75652ad34e55b571eb61b5d9c7bc: Time spent bootstrapping tablet: real 0.196s user 0.188s sys 0.007s
I20240502 07:58:46.375700 8974 raft_consensus.cc:348] T 18a448dbe2af493abefbc728a8f85186 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 312 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false replace: true } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } }
I20240502 07:58:46.375844 8974 raft_consensus.cc:729] T 18a448dbe2af493abefbc728a8f85186 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 389b75652ad34e55b571eb61b5d9c7bc, State: Initialized, Role: LEARNER
I20240502 07:58:46.375957 8974 consensus_queue.cc:260] T 18a448dbe2af493abefbc728a8f85186 P 389b75652ad34e55b571eb61b5d9c7bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 312, Last appended: 1.312, Last appended by leader: 312, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 312 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false replace: true } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } }
I20240502 07:58:46.376240 8974 ts_tablet_manager.cc:1398] T 18a448dbe2af493abefbc728a8f85186 P 389b75652ad34e55b571eb61b5d9c7bc: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:46.376729 5650 tablet_copy_service.cc:342] P 91e82003a656442189a329b29846a841: Request end of tablet copy session 389b75652ad34e55b571eb61b5d9c7bc-18a448dbe2af493abefbc728a8f85186 received from {username='slave'} at 127.3.139.72:41667
I20240502 07:58:46.376804 5650 tablet_copy_service.cc:433] P 91e82003a656442189a329b29846a841: ending tablet copy session 389b75652ad34e55b571eb61b5d9c7bc-18a448dbe2af493abefbc728a8f85186 on tablet 18a448dbe2af493abefbc728a8f85186 with peer 389b75652ad34e55b571eb61b5d9c7bc
I20240502 07:58:46.411586 8974 ts_tablet_manager.cc:902] T 635b0a1a0fd64eeeac70025014c9ab81 P 389b75652ad34e55b571eb61b5d9c7bc: Initiating tablet copy from peer eda55a8566c447cfa749457825b70688 (127.3.139.66:39925)
I20240502 07:58:46.411962 8974 tablet_copy_client.cc:308] T 635b0a1a0fd64eeeac70025014c9ab81 P 389b75652ad34e55b571eb61b5d9c7bc: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.66:39925
I20240502 07:58:46.412389 5781 tablet_copy_service.cc:140] P eda55a8566c447cfa749457825b70688: Received BeginTabletCopySession request for tablet 635b0a1a0fd64eeeac70025014c9ab81 from peer 389b75652ad34e55b571eb61b5d9c7bc ({username='slave'} at 127.3.139.72:53673)
I20240502 07:58:46.412482 5781 tablet_copy_service.cc:161] P eda55a8566c447cfa749457825b70688: Beginning new tablet copy session on tablet 635b0a1a0fd64eeeac70025014c9ab81 from peer 389b75652ad34e55b571eb61b5d9c7bc at {username='slave'} at 127.3.139.72:53673: session id = 389b75652ad34e55b571eb61b5d9c7bc-635b0a1a0fd64eeeac70025014c9ab81
I20240502 07:58:46.413201 5781 tablet_copy_source_session.cc:205] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:46.414001 8974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 635b0a1a0fd64eeeac70025014c9ab81. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:46.415690 8974 tablet_copy_client.cc:783] T 635b0a1a0fd64eeeac70025014c9ab81 P 389b75652ad34e55b571eb61b5d9c7bc: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:46.415831 8974 tablet_copy_client.cc:647] T 635b0a1a0fd64eeeac70025014c9ab81 P 389b75652ad34e55b571eb61b5d9c7bc: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:46.417304 8974 tablet_copy_client.cc:523] T 635b0a1a0fd64eeeac70025014c9ab81 P 389b75652ad34e55b571eb61b5d9c7bc: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:46.418202 8974 tablet_bootstrap.cc:492] T 635b0a1a0fd64eeeac70025014c9ab81 P 389b75652ad34e55b571eb61b5d9c7bc: Bootstrap starting.
I20240502 07:58:46.438864 8967 ts_tablet_manager.cc:902] T d41a2cb7b69e40d9a681384e5ad0a451 P 389b75652ad34e55b571eb61b5d9c7bc: Initiating tablet copy from peer d58f6d1ff8cf472f8a9edf3700e64bfb (127.3.139.70:41047)
I20240502 07:58:46.439395 8967 tablet_copy_client.cc:308] T d41a2cb7b69e40d9a681384e5ad0a451 P 389b75652ad34e55b571eb61b5d9c7bc: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.70:41047
I20240502 07:58:46.439911 6303 tablet_copy_service.cc:140] P d58f6d1ff8cf472f8a9edf3700e64bfb: Received BeginTabletCopySession request for tablet d41a2cb7b69e40d9a681384e5ad0a451 from peer 389b75652ad34e55b571eb61b5d9c7bc ({username='slave'} at 127.3.139.72:35179)
I20240502 07:58:46.440004 6303 tablet_copy_service.cc:161] P d58f6d1ff8cf472f8a9edf3700e64bfb: Beginning new tablet copy session on tablet d41a2cb7b69e40d9a681384e5ad0a451 from peer 389b75652ad34e55b571eb61b5d9c7bc at {username='slave'} at 127.3.139.72:35179: session id = 389b75652ad34e55b571eb61b5d9c7bc-d41a2cb7b69e40d9a681384e5ad0a451
I20240502 07:58:46.440749 6303 tablet_copy_source_session.cc:205] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:46.441442 8967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d41a2cb7b69e40d9a681384e5ad0a451. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:46.442934 8967 tablet_copy_client.cc:783] T d41a2cb7b69e40d9a681384e5ad0a451 P 389b75652ad34e55b571eb61b5d9c7bc: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:46.443099 8967 tablet_copy_client.cc:647] T d41a2cb7b69e40d9a681384e5ad0a451 P 389b75652ad34e55b571eb61b5d9c7bc: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:46.444979 8967 tablet_copy_client.cc:523] T d41a2cb7b69e40d9a681384e5ad0a451 P 389b75652ad34e55b571eb61b5d9c7bc: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:46.445879 8967 tablet_bootstrap.cc:492] T d41a2cb7b69e40d9a681384e5ad0a451 P 389b75652ad34e55b571eb61b5d9c7bc: Bootstrap starting.
I20240502 07:58:46.572588 8322 raft_consensus.cc:1203] T 18a448dbe2af493abefbc728a8f85186 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 LEARNER]: Deduplicated request from leader. Original: 1.311->[1.312-1.312] Dedup: 1.312->[]
I20240502 07:58:46.598821 8974 tablet_bootstrap.cc:492] T 635b0a1a0fd64eeeac70025014c9ab81 P 389b75652ad34e55b571eb61b5d9c7bc: Bootstrap replayed 1/1 log segments. Stats: ops{read=293 overwritten=0 applied=293 ignored=0} inserts{seen=965 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:58:46.599102 8974 tablet_bootstrap.cc:492] T 635b0a1a0fd64eeeac70025014c9ab81 P 389b75652ad34e55b571eb61b5d9c7bc: Bootstrap complete.
I20240502 07:58:46.599295 8974 ts_tablet_manager.cc:1367] T 635b0a1a0fd64eeeac70025014c9ab81 P 389b75652ad34e55b571eb61b5d9c7bc: Time spent bootstrapping tablet: real 0.181s user 0.175s sys 0.006s
I20240502 07:58:46.599678 8974 raft_consensus.cc:348] T 635b0a1a0fd64eeeac70025014c9ab81 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 293 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } }
I20240502 07:58:46.599792 8974 raft_consensus.cc:729] T 635b0a1a0fd64eeeac70025014c9ab81 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 389b75652ad34e55b571eb61b5d9c7bc, State: Initialized, Role: LEARNER
I20240502 07:58:46.599884 8974 consensus_queue.cc:260] T 635b0a1a0fd64eeeac70025014c9ab81 P 389b75652ad34e55b571eb61b5d9c7bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 293, Last appended: 1.293, Last appended by leader: 293, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 293 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } }
I20240502 07:58:46.600139 8974 ts_tablet_manager.cc:1398] T 635b0a1a0fd64eeeac70025014c9ab81 P 389b75652ad34e55b571eb61b5d9c7bc: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:46.600708 5781 tablet_copy_service.cc:342] P eda55a8566c447cfa749457825b70688: Request end of tablet copy session 389b75652ad34e55b571eb61b5d9c7bc-635b0a1a0fd64eeeac70025014c9ab81 received from {username='slave'} at 127.3.139.72:53673
I20240502 07:58:46.600786 5781 tablet_copy_service.cc:433] P eda55a8566c447cfa749457825b70688: ending tablet copy session 389b75652ad34e55b571eb61b5d9c7bc-635b0a1a0fd64eeeac70025014c9ab81 on tablet 635b0a1a0fd64eeeac70025014c9ab81 with peer 389b75652ad34e55b571eb61b5d9c7bc
I20240502 07:58:46.610916 8967 tablet_bootstrap.cc:492] T d41a2cb7b69e40d9a681384e5ad0a451 P 389b75652ad34e55b571eb61b5d9c7bc: Bootstrap replayed 1/1 log segments. Stats: ops{read=279 overwritten=0 applied=279 ignored=0} inserts{seen=958 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:58:46.611179 8967 tablet_bootstrap.cc:492] T d41a2cb7b69e40d9a681384e5ad0a451 P 389b75652ad34e55b571eb61b5d9c7bc: Bootstrap complete.
I20240502 07:58:46.611320 8967 ts_tablet_manager.cc:1367] T d41a2cb7b69e40d9a681384e5ad0a451 P 389b75652ad34e55b571eb61b5d9c7bc: Time spent bootstrapping tablet: real 0.165s user 0.147s sys 0.017s
I20240502 07:58:46.611647 8967 raft_consensus.cc:348] T d41a2cb7b69e40d9a681384e5ad0a451 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 279 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } }
I20240502 07:58:46.611742 8967 raft_consensus.cc:729] T d41a2cb7b69e40d9a681384e5ad0a451 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 389b75652ad34e55b571eb61b5d9c7bc, State: Initialized, Role: LEARNER
I20240502 07:58:46.611807 8967 consensus_queue.cc:260] T d41a2cb7b69e40d9a681384e5ad0a451 P 389b75652ad34e55b571eb61b5d9c7bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 279, Last appended: 1.279, Last appended by leader: 279, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 279 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } }
I20240502 07:58:46.612025 8967 ts_tablet_manager.cc:1398] T d41a2cb7b69e40d9a681384e5ad0a451 P 389b75652ad34e55b571eb61b5d9c7bc: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:46.612470 6303 tablet_copy_service.cc:342] P d58f6d1ff8cf472f8a9edf3700e64bfb: Request end of tablet copy session 389b75652ad34e55b571eb61b5d9c7bc-d41a2cb7b69e40d9a681384e5ad0a451 received from {username='slave'} at 127.3.139.72:35179
I20240502 07:58:46.612555 6303 tablet_copy_service.cc:433] P d58f6d1ff8cf472f8a9edf3700e64bfb: ending tablet copy session 389b75652ad34e55b571eb61b5d9c7bc-d41a2cb7b69e40d9a681384e5ad0a451 on tablet d41a2cb7b69e40d9a681384e5ad0a451 with peer 389b75652ad34e55b571eb61b5d9c7bc
I20240502 07:58:46.616708 8967 ts_tablet_manager.cc:902] T 96dcb06388374514951f24393f288eca P 389b75652ad34e55b571eb61b5d9c7bc: Initiating tablet copy from peer 91e82003a656442189a329b29846a841 (127.3.139.65:43429)
I20240502 07:58:46.617069 8967 tablet_copy_client.cc:308] T 96dcb06388374514951f24393f288eca P 389b75652ad34e55b571eb61b5d9c7bc: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.65:43429
I20240502 07:58:46.617441 5650 tablet_copy_service.cc:140] P 91e82003a656442189a329b29846a841: Received BeginTabletCopySession request for tablet 96dcb06388374514951f24393f288eca from peer 389b75652ad34e55b571eb61b5d9c7bc ({username='slave'} at 127.3.139.72:41667)
I20240502 07:58:46.617547 5650 tablet_copy_service.cc:161] P 91e82003a656442189a329b29846a841: Beginning new tablet copy session on tablet 96dcb06388374514951f24393f288eca from peer 389b75652ad34e55b571eb61b5d9c7bc at {username='slave'} at 127.3.139.72:41667: session id = 389b75652ad34e55b571eb61b5d9c7bc-96dcb06388374514951f24393f288eca
I20240502 07:58:46.618261 5650 tablet_copy_source_session.cc:205] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:46.618896 8967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96dcb06388374514951f24393f288eca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:46.620407 8967 tablet_copy_client.cc:783] T 96dcb06388374514951f24393f288eca P 389b75652ad34e55b571eb61b5d9c7bc: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:46.620549 8967 tablet_copy_client.cc:647] T 96dcb06388374514951f24393f288eca P 389b75652ad34e55b571eb61b5d9c7bc: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:46.622292 8967 tablet_copy_client.cc:523] T 96dcb06388374514951f24393f288eca P 389b75652ad34e55b571eb61b5d9c7bc: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:46.623203 8967 tablet_bootstrap.cc:492] T 96dcb06388374514951f24393f288eca P 389b75652ad34e55b571eb61b5d9c7bc: Bootstrap starting.
I20240502 07:58:46.701700 8322 raft_consensus.cc:1203] T eede21c622d8435ba1efa473ef97b1d1 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 LEARNER]: Deduplicated request from leader. Original: 1.166->[1.167-1.167] Dedup: 1.167->[]
I20240502 07:58:46.898450 8967 tablet_bootstrap.cc:492] T 96dcb06388374514951f24393f288eca P 389b75652ad34e55b571eb61b5d9c7bc: Bootstrap replayed 1/1 log segments. Stats: ops{read=501 overwritten=0 applied=501 ignored=0} inserts{seen=1180 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:58:46.898699 8967 tablet_bootstrap.cc:492] T 96dcb06388374514951f24393f288eca P 389b75652ad34e55b571eb61b5d9c7bc: Bootstrap complete.
I20240502 07:58:46.898873 8967 ts_tablet_manager.cc:1367] T 96dcb06388374514951f24393f288eca P 389b75652ad34e55b571eb61b5d9c7bc: Time spent bootstrapping tablet: real 0.276s user 0.256s sys 0.016s
I20240502 07:58:46.899261 8967 raft_consensus.cc:348] T 96dcb06388374514951f24393f288eca P 389b75652ad34e55b571eb61b5d9c7bc [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 501 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } }
I20240502 07:58:46.899377 8967 raft_consensus.cc:729] T 96dcb06388374514951f24393f288eca P 389b75652ad34e55b571eb61b5d9c7bc [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 389b75652ad34e55b571eb61b5d9c7bc, State: Initialized, Role: LEARNER
I20240502 07:58:46.899456 8967 consensus_queue.cc:260] T 96dcb06388374514951f24393f288eca P 389b75652ad34e55b571eb61b5d9c7bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 501, Last appended: 1.501, Last appended by leader: 501, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 501 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: true } }
I20240502 07:58:46.899700 8967 ts_tablet_manager.cc:1398] T 96dcb06388374514951f24393f288eca P 389b75652ad34e55b571eb61b5d9c7bc: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240502 07:58:46.900182 5650 tablet_copy_service.cc:342] P 91e82003a656442189a329b29846a841: Request end of tablet copy session 389b75652ad34e55b571eb61b5d9c7bc-96dcb06388374514951f24393f288eca received from {username='slave'} at 127.3.139.72:41667
I20240502 07:58:46.900269 5650 tablet_copy_service.cc:433] P 91e82003a656442189a329b29846a841: ending tablet copy session 389b75652ad34e55b571eb61b5d9c7bc-96dcb06388374514951f24393f288eca on tablet 96dcb06388374514951f24393f288eca with peer 389b75652ad34e55b571eb61b5d9c7bc
I20240502 07:58:46.975353 8322 raft_consensus.cc:1203] T d41a2cb7b69e40d9a681384e5ad0a451 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 LEARNER]: Deduplicated request from leader. Original: 1.278->[1.279-1.279] Dedup: 1.279->[]
I20240502 07:58:46.977542 8322 raft_consensus.cc:1203] T 635b0a1a0fd64eeeac70025014c9ab81 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 LEARNER]: Deduplicated request from leader. Original: 1.292->[1.293-1.293] Dedup: 1.293->[]
I20240502 07:58:47.008610 8251 raft_consensus.cc:1050] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841: attempting to promote NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc to VOTER
I20240502 07:58:47.009101 8251 consensus_queue.cc:237] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 312, Committed index: 312, Last appended: 1.312, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 313 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false replace: true } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } }
I20240502 07:58:47.010598 7044 raft_consensus.cc:1261] T 18a448dbe2af493abefbc728a8f85186 P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 312. Preceding OpId from leader: term: 1 index: 313. (index mismatch)
I20240502 07:58:47.010613 6151 raft_consensus.cc:1261] T 18a448dbe2af493abefbc728a8f85186 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 312. Preceding OpId from leader: term: 1 index: 313. (index mismatch)
I20240502 07:58:47.010717 5892 raft_consensus.cc:1261] T 18a448dbe2af493abefbc728a8f85186 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 312. Preceding OpId from leader: term: 1 index: 313. (index mismatch)
I20240502 07:58:47.011014 6018 raft_consensus.cc:1261] T 18a448dbe2af493abefbc728a8f85186 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 312. Preceding OpId from leader: term: 1 index: 313. (index mismatch)
I20240502 07:58:47.011150 9099 consensus_queue.cc:1035] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 313, Last known committed idx: 312, Time since last communication: 0.000s
I20240502 07:58:47.011458 9100 consensus_queue.cc:1035] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 313, Last known committed idx: 312, Time since last communication: 0.000s
I20240502 07:58:47.011605 8322 raft_consensus.cc:1261] T 18a448dbe2af493abefbc728a8f85186 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 LEARNER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 312. Preceding OpId from leader: term: 1 index: 313. (index mismatch)
I20240502 07:58:47.012216 9100 consensus_queue.cc:1035] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 313, Last known committed idx: 312, Time since last communication: 0.000s
I20240502 07:58:47.012432 9099 consensus_queue.cc:1035] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 313, Last known committed idx: 312, Time since last communication: 0.000s
I20240502 07:58:47.012702 9099 consensus_queue.cc:1035] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 313, Last known committed idx: 312, Time since last communication: 0.000s
I20240502 07:58:47.015084 8322 raft_consensus.cc:2932] T 18a448dbe2af493abefbc728a8f85186 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 FOLLOWER]: Committing config change with OpId 1.313: config changed from index 312 to 313, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 313 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false replace: true } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.015537 7044 raft_consensus.cc:2932] T 18a448dbe2af493abefbc728a8f85186 P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Committing config change with OpId 1.313: config changed from index 312 to 313, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 313 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false replace: true } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.014619 9100 raft_consensus.cc:2932] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [term 1 LEADER]: Committing config change with OpId 1.313: config changed from index 312 to 313, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 313 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false replace: true } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.017122 6151 raft_consensus.cc:2932] T 18a448dbe2af493abefbc728a8f85186 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.313: config changed from index 312 to 313, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 313 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false replace: true } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.017508 5483 catalog_manager.cc:5484] T 18a448dbe2af493abefbc728a8f85186 P 389b75652ad34e55b571eb61b5d9c7bc reported cstate change: config changed from index 312 to 313, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "91e82003a656442189a329b29846a841" committed_config { opid_index: 313 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false replace: true } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.019780 5892 raft_consensus.cc:2932] T 18a448dbe2af493abefbc728a8f85186 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Committing config change with OpId 1.313: config changed from index 312 to 313, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 313 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false replace: true } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.019935 6018 raft_consensus.cc:2932] T 18a448dbe2af493abefbc728a8f85186 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Committing config change with OpId 1.313: config changed from index 312 to 313, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 313 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false replace: true } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.023975 5628 consensus_queue.cc:237] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 313, Committed index: 313, Last appended: 1.313, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 314 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } }
I20240502 07:58:47.025086 6151 raft_consensus.cc:1261] T 18a448dbe2af493abefbc728a8f85186 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 313. Preceding OpId from leader: term: 1 index: 314. (index mismatch)
I20240502 07:58:47.025158 8322 raft_consensus.cc:1261] T 18a448dbe2af493abefbc728a8f85186 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 313. Preceding OpId from leader: term: 1 index: 314. (index mismatch)
I20240502 07:58:47.025177 6018 raft_consensus.cc:1261] T 18a448dbe2af493abefbc728a8f85186 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 313. Preceding OpId from leader: term: 1 index: 314. (index mismatch)
I20240502 07:58:47.025483 9101 consensus_queue.cc:1035] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 314, Last known committed idx: 313, Time since last communication: 0.000s
I20240502 07:58:47.025643 8787 consensus_queue.cc:1035] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 314, Last known committed idx: 313, Time since last communication: 0.000s
I20240502 07:58:47.025732 5892 raft_consensus.cc:1261] T 18a448dbe2af493abefbc728a8f85186 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 313. Preceding OpId from leader: term: 1 index: 314. (index mismatch)
I20240502 07:58:47.026000 9099 consensus_queue.cc:1035] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 314, Last known committed idx: 313, Time since last communication: 0.000s
I20240502 07:58:47.026242 9101 consensus_queue.cc:1035] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 314, Last known committed idx: 313, Time since last communication: 0.000s
I20240502 07:58:47.028225 6151 raft_consensus.cc:2932] T 18a448dbe2af493abefbc728a8f85186 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.314: config changed from index 313 to 314, VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) evicted. New config: { opid_index: 314 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.028373 6018 raft_consensus.cc:2932] T 18a448dbe2af493abefbc728a8f85186 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Committing config change with OpId 1.314: config changed from index 313 to 314, VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) evicted. New config: { opid_index: 314 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.028645 5892 raft_consensus.cc:2932] T 18a448dbe2af493abefbc728a8f85186 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Committing config change with OpId 1.314: config changed from index 313 to 314, VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) evicted. New config: { opid_index: 314 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.030288 5480 catalog_manager.cc:5484] T 18a448dbe2af493abefbc728a8f85186 P e020e7ffe8ce423a8bdefec667161217 reported cstate change: config changed from index 313 to 314, VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) evicted. New cstate: current_term: 1 leader_uuid: "91e82003a656442189a329b29846a841" committed_config { opid_index: 314 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.029958 9100 raft_consensus.cc:2932] T 18a448dbe2af493abefbc728a8f85186 P 91e82003a656442189a329b29846a841 [term 1 LEADER]: Committing config change with OpId 1.314: config changed from index 313 to 314, VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) evicted. New config: { opid_index: 314 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.032044 5473 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 18a448dbe2af493abefbc728a8f85186 with cas_config_opid_index 313: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:58:47.032630 8322 raft_consensus.cc:2932] T 18a448dbe2af493abefbc728a8f85186 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 FOLLOWER]: Committing config change with OpId 1.314: config changed from index 313 to 314, VOTER 27e3fafad7a24bcda362d2197be87017 (127.3.139.71) evicted. New config: { opid_index: 314 OBSOLETE_local: false peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.037693 7028 tablet_service.cc:1514] Processing DeleteTablet for tablet 18a448dbe2af493abefbc728a8f85186 with delete_type TABLET_DATA_TOMBSTONED (TS 27e3fafad7a24bcda362d2197be87017 not found in new config with opid_index 314) from {username='slave'} at 127.0.0.1:56760
I20240502 07:58:47.038079 9113 tablet_replica.cc:330] T 18a448dbe2af493abefbc728a8f85186 P 27e3fafad7a24bcda362d2197be87017: stopping tablet replica
I20240502 07:58:47.038340 9113 raft_consensus.cc:2229] T 18a448dbe2af493abefbc728a8f85186 P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:47.038573 9113 raft_consensus.cc:2258] T 18a448dbe2af493abefbc728a8f85186 P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:47.039958 9113 ts_tablet_manager.cc:1875] T 18a448dbe2af493abefbc728a8f85186 P 27e3fafad7a24bcda362d2197be87017: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:58:47.041759 9113 ts_tablet_manager.cc:1888] T 18a448dbe2af493abefbc728a8f85186 P 27e3fafad7a24bcda362d2197be87017: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.313
I20240502 07:58:47.041852 9113 log.cc:1196] T 18a448dbe2af493abefbc728a8f85186 P 27e3fafad7a24bcda362d2197be87017: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/wal/wals/18a448dbe2af493abefbc728a8f85186
I20240502 07:58:47.042361 5474 catalog_manager.cc:4830] TS 27e3fafad7a24bcda362d2197be87017 (127.3.139.71:35341): tablet 18a448dbe2af493abefbc728a8f85186 (table rebalance_test_table_2 [id=28039a31ea894d23af67c1f1da387f21]) successfully deleted
I20240502 07:58:47.128199 8322 raft_consensus.cc:1203] T 96dcb06388374514951f24393f288eca P 389b75652ad34e55b571eb61b5d9c7bc [term 1 LEARNER]: Deduplicated request from leader. Original: 1.500->[1.501-1.501] Dedup: 1.501->[]
I20240502 07:58:47.136397 9100 raft_consensus.cc:1050] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841: attempting to promote NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc to VOTER
I20240502 07:58:47.136763 9100 consensus_queue.cc:237] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 167, Committed index: 167, Last appended: 1.167, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 168 OBSOLETE_local: false peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } }
I20240502 07:58:47.138173 8322 raft_consensus.cc:1261] T eede21c622d8435ba1efa473ef97b1d1 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 LEARNER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 167. Preceding OpId from leader: term: 1 index: 168. (index mismatch)
I20240502 07:58:47.138188 5892 raft_consensus.cc:1261] T eede21c622d8435ba1efa473ef97b1d1 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 167. Preceding OpId from leader: term: 1 index: 168. (index mismatch)
I20240502 07:58:47.138243 6018 raft_consensus.cc:1261] T eede21c622d8435ba1efa473ef97b1d1 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 167. Preceding OpId from leader: term: 1 index: 168. (index mismatch)
I20240502 07:58:47.138540 9100 consensus_queue.cc:1035] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 168, Last known committed idx: 167, Time since last communication: 0.000s
I20240502 07:58:47.138639 5760 raft_consensus.cc:1261] T eede21c622d8435ba1efa473ef97b1d1 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 167. Preceding OpId from leader: term: 1 index: 168. (index mismatch)
I20240502 07:58:47.138705 6151 raft_consensus.cc:1261] T eede21c622d8435ba1efa473ef97b1d1 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 167. Preceding OpId from leader: term: 1 index: 168. (index mismatch)
I20240502 07:58:47.138882 9100 consensus_queue.cc:1035] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 168, Last known committed idx: 167, Time since last communication: 0.000s
I20240502 07:58:47.139012 8787 consensus_queue.cc:1035] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 168, Last known committed idx: 167, Time since last communication: 0.000s
I20240502 07:58:47.139194 9101 consensus_queue.cc:1035] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 168, Last known committed idx: 167, Time since last communication: 0.000s
I20240502 07:58:47.139367 8251 consensus_queue.cc:1035] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 168, Last known committed idx: 167, Time since last communication: 0.000s
I20240502 07:58:47.141748 8787 raft_consensus.cc:2932] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841 [term 1 LEADER]: Committing config change with OpId 1.168: config changed from index 167 to 168, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 168 OBSOLETE_local: false peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.142105 5760 raft_consensus.cc:2932] T eede21c622d8435ba1efa473ef97b1d1 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Committing config change with OpId 1.168: config changed from index 167 to 168, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 168 OBSOLETE_local: false peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.142390 5892 raft_consensus.cc:2932] T eede21c622d8435ba1efa473ef97b1d1 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Committing config change with OpId 1.168: config changed from index 167 to 168, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 168 OBSOLETE_local: false peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.143541 8322 raft_consensus.cc:2932] T eede21c622d8435ba1efa473ef97b1d1 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 FOLLOWER]: Committing config change with OpId 1.168: config changed from index 167 to 168, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 168 OBSOLETE_local: false peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.143857 6151 raft_consensus.cc:2932] T eede21c622d8435ba1efa473ef97b1d1 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.168: config changed from index 167 to 168, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 168 OBSOLETE_local: false peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.147668 5480 catalog_manager.cc:5484] T eede21c622d8435ba1efa473ef97b1d1 P 389b75652ad34e55b571eb61b5d9c7bc reported cstate change: config changed from index 167 to 168, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "91e82003a656442189a329b29846a841" committed_config { opid_index: 168 OBSOLETE_local: false peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.148336 6018 raft_consensus.cc:2932] T eede21c622d8435ba1efa473ef97b1d1 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Committing config change with OpId 1.168: config changed from index 167 to 168, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 168 OBSOLETE_local: false peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "d5bc7863cd294f77a932c4a40f98b79d" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 44287 } attrs { replace: true } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.151772 5628 consensus_queue.cc:237] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 168, Committed index: 168, Last appended: 1.168, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 169 OBSOLETE_local: false peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } }
I20240502 07:58:47.152953 5760 raft_consensus.cc:1261] T eede21c622d8435ba1efa473ef97b1d1 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 168. Preceding OpId from leader: term: 1 index: 169. (index mismatch)
I20240502 07:58:47.153106 8322 raft_consensus.cc:1261] T eede21c622d8435ba1efa473ef97b1d1 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 168. Preceding OpId from leader: term: 1 index: 169. (index mismatch)
I20240502 07:58:47.153184 6151 raft_consensus.cc:1261] T eede21c622d8435ba1efa473ef97b1d1 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 168. Preceding OpId from leader: term: 1 index: 169. (index mismatch)
I20240502 07:58:47.153349 9100 consensus_queue.cc:1035] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 169, Last known committed idx: 168, Time since last communication: 0.000s
I20240502 07:58:47.153539 9101 consensus_queue.cc:1035] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 169, Last known committed idx: 168, Time since last communication: 0.000s
I20240502 07:58:47.153693 8251 consensus_queue.cc:1035] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 169, Last known committed idx: 168, Time since last communication: 0.000s
I20240502 07:58:47.153743 5892 raft_consensus.cc:1261] T eede21c622d8435ba1efa473ef97b1d1 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 168. Preceding OpId from leader: term: 1 index: 169. (index mismatch)
I20240502 07:58:47.154138 8251 consensus_queue.cc:1035] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 169, Last known committed idx: 168, Time since last communication: 0.000s
I20240502 07:58:47.155346 8251 raft_consensus.cc:2932] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841 [term 1 LEADER]: Committing config change with OpId 1.169: config changed from index 168 to 169, VOTER d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68) evicted. New config: { opid_index: 169 OBSOLETE_local: false peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.155704 6151 raft_consensus.cc:2932] T eede21c622d8435ba1efa473ef97b1d1 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.169: config changed from index 168 to 169, VOTER d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68) evicted. New config: { opid_index: 169 OBSOLETE_local: false peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.157014 5473 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet eede21c622d8435ba1efa473ef97b1d1 with cas_config_opid_index 168: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:58:47.157503 8322 raft_consensus.cc:2932] T eede21c622d8435ba1efa473ef97b1d1 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 FOLLOWER]: Committing config change with OpId 1.169: config changed from index 168 to 169, VOTER d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68) evicted. New config: { opid_index: 169 OBSOLETE_local: false peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.158129 5480 catalog_manager.cc:5484] T eede21c622d8435ba1efa473ef97b1d1 P 91e82003a656442189a329b29846a841 reported cstate change: config changed from index 168 to 169, VOTER d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68) evicted. New cstate: current_term: 1 leader_uuid: "91e82003a656442189a329b29846a841" committed_config { opid_index: 169 OBSOLETE_local: false peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:58:47.158313 5891 raft_consensus.cc:2932] T eede21c622d8435ba1efa473ef97b1d1 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Committing config change with OpId 1.169: config changed from index 168 to 169, VOTER d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68) evicted. New config: { opid_index: 169 OBSOLETE_local: false peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.160732 5760 raft_consensus.cc:2932] T eede21c622d8435ba1efa473ef97b1d1 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Committing config change with OpId 1.169: config changed from index 168 to 169, VOTER d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68) evicted. New config: { opid_index: 169 OBSOLETE_local: false peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.162557 6002 tablet_service.cc:1514] Processing DeleteTablet for tablet eede21c622d8435ba1efa473ef97b1d1 with delete_type TABLET_DATA_TOMBSTONED (TS d5bc7863cd294f77a932c4a40f98b79d not found in new config with opid_index 169) from {username='slave'} at 127.0.0.1:39270
I20240502 07:58:47.168926 9120 tablet_replica.cc:330] T eede21c622d8435ba1efa473ef97b1d1 P d5bc7863cd294f77a932c4a40f98b79d: stopping tablet replica
I20240502 07:58:47.169050 9120 raft_consensus.cc:2229] T eede21c622d8435ba1efa473ef97b1d1 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:47.169183 9120 raft_consensus.cc:2258] T eede21c622d8435ba1efa473ef97b1d1 P d5bc7863cd294f77a932c4a40f98b79d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:47.170272 9120 ts_tablet_manager.cc:1875] T eede21c622d8435ba1efa473ef97b1d1 P d5bc7863cd294f77a932c4a40f98b79d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:58:47.172159 9120 ts_tablet_manager.cc:1888] T eede21c622d8435ba1efa473ef97b1d1 P d5bc7863cd294f77a932c4a40f98b79d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.168
I20240502 07:58:47.172222 9120 log.cc:1196] T eede21c622d8435ba1efa473ef97b1d1 P d5bc7863cd294f77a932c4a40f98b79d: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal/wals/eede21c622d8435ba1efa473ef97b1d1
I20240502 07:58:47.172657 5473 catalog_manager.cc:4830] TS d5bc7863cd294f77a932c4a40f98b79d (127.3.139.68:44287): tablet eede21c622d8435ba1efa473ef97b1d1 (table rebalance_test_table_4 [id=5c1e49d6edf446d5a06063e568da2a0a]) successfully deleted
I20240502 07:58:47.435228 9072 raft_consensus.cc:1050] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb: attempting to promote NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc to VOTER
I20240502 07:58:47.435622 9072 consensus_queue.cc:237] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 279, Committed index: 279, Last appended: 1.279, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 280 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } }
I20240502 07:58:47.437003 7044 raft_consensus.cc:1261] T d41a2cb7b69e40d9a681384e5ad0a451 P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 279. Preceding OpId from leader: term: 1 index: 280. (index mismatch)
I20240502 07:58:47.437148 5760 raft_consensus.cc:1261] T d41a2cb7b69e40d9a681384e5ad0a451 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 279. Preceding OpId from leader: term: 1 index: 280. (index mismatch)
I20240502 07:58:47.437156 6151 raft_consensus.cc:1261] T d41a2cb7b69e40d9a681384e5ad0a451 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 279. Preceding OpId from leader: term: 1 index: 280. (index mismatch)
I20240502 07:58:47.437175 8322 raft_consensus.cc:1261] T d41a2cb7b69e40d9a681384e5ad0a451 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 LEARNER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 279. Preceding OpId from leader: term: 1 index: 280. (index mismatch)
I20240502 07:58:47.437503 5627 raft_consensus.cc:1261] T d41a2cb7b69e40d9a681384e5ad0a451 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 279. Preceding OpId from leader: term: 1 index: 280. (index mismatch)
I20240502 07:58:47.437479 9162 consensus_queue.cc:1035] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 280, Last known committed idx: 279, Time since last communication: 0.000s
I20240502 07:58:47.437641 9164 consensus_queue.cc:1035] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 280, Last known committed idx: 279, Time since last communication: 0.000s
I20240502 07:58:47.437793 8732 consensus_queue.cc:1035] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 280, Last known committed idx: 279, Time since last communication: 0.000s
I20240502 07:58:47.437933 9072 consensus_queue.cc:1035] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 280, Last known committed idx: 279, Time since last communication: 0.000s
I20240502 07:58:47.438323 9163 consensus_queue.cc:1035] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 280, Last known committed idx: 279, Time since last communication: 0.000s
I20240502 07:58:47.440382 9164 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 LEADER]: Committing config change with OpId 1.280: config changed from index 279 to 280, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 280 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.440829 6151 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.280: config changed from index 279 to 280, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 280 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.440992 8322 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 FOLLOWER]: Committing config change with OpId 1.280: config changed from index 279 to 280, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 280 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.441982 5627 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Committing config change with OpId 1.280: config changed from index 279 to 280, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 280 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.443565 7044 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Committing config change with OpId 1.280: config changed from index 279 to 280, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 280 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.443876 5484 catalog_manager.cc:5484] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb reported cstate change: config changed from index 279 to 280, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" committed_config { opid_index: 280 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:58:47.444010 5760 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Committing config change with OpId 1.280: config changed from index 279 to 280, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 280 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.447440 6281 consensus_queue.cc:237] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 280, Committed index: 280, Last appended: 1.280, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 281 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } }
I20240502 07:58:47.448872 6151 raft_consensus.cc:1261] T d41a2cb7b69e40d9a681384e5ad0a451 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 280. Preceding OpId from leader: term: 1 index: 281. (index mismatch)
I20240502 07:58:47.448897 8322 raft_consensus.cc:1261] T d41a2cb7b69e40d9a681384e5ad0a451 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 280. Preceding OpId from leader: term: 1 index: 281. (index mismatch)
I20240502 07:58:47.448966 5760 raft_consensus.cc:1261] T d41a2cb7b69e40d9a681384e5ad0a451 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 280. Preceding OpId from leader: term: 1 index: 281. (index mismatch)
I20240502 07:58:47.449297 8732 consensus_queue.cc:1035] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 281, Last known committed idx: 280, Time since last communication: 0.000s
I20240502 07:58:47.449359 7044 raft_consensus.cc:1261] T d41a2cb7b69e40d9a681384e5ad0a451 P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Refusing update from remote peer d58f6d1ff8cf472f8a9edf3700e64bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 280. Preceding OpId from leader: term: 1 index: 281. (index mismatch)
I20240502 07:58:47.449434 9162 consensus_queue.cc:1035] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 281, Last known committed idx: 280, Time since last communication: 0.000s
I20240502 07:58:47.449648 9164 consensus_queue.cc:1035] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 281, Last known committed idx: 280, Time since last communication: 0.000s
I20240502 07:58:47.449771 9072 consensus_queue.cc:1035] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 281, Last known committed idx: 280, Time since last communication: 0.000s
I20240502 07:58:47.451345 9072 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 LEADER]: Committing config change with OpId 1.281: config changed from index 280 to 281, VOTER 91e82003a656442189a329b29846a841 (127.3.139.65) evicted. New config: { opid_index: 281 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.451802 6151 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.281: config changed from index 280 to 281, VOTER 91e82003a656442189a329b29846a841 (127.3.139.65) evicted. New config: { opid_index: 281 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.452867 5474 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet d41a2cb7b69e40d9a681384e5ad0a451 with cas_config_opid_index 280: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:58:47.453459 7044 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Committing config change with OpId 1.281: config changed from index 280 to 281, VOTER 91e82003a656442189a329b29846a841 (127.3.139.65) evicted. New config: { opid_index: 281 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.453524 8322 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 FOLLOWER]: Committing config change with OpId 1.281: config changed from index 280 to 281, VOTER 91e82003a656442189a329b29846a841 (127.3.139.65) evicted. New config: { opid_index: 281 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.453759 5480 catalog_manager.cc:5484] T d41a2cb7b69e40d9a681384e5ad0a451 P d58f6d1ff8cf472f8a9edf3700e64bfb reported cstate change: config changed from index 280 to 281, VOTER 91e82003a656442189a329b29846a841 (127.3.139.65) evicted. New cstate: current_term: 1 leader_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" committed_config { opid_index: 281 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:58:47.456353 5608 tablet_service.cc:1514] Processing DeleteTablet for tablet d41a2cb7b69e40d9a681384e5ad0a451 with delete_type TABLET_DATA_TOMBSTONED (TS 91e82003a656442189a329b29846a841 not found in new config with opid_index 281) from {username='slave'} at 127.0.0.1:54448
I20240502 07:58:47.456687 9174 tablet_replica.cc:330] T d41a2cb7b69e40d9a681384e5ad0a451 P 91e82003a656442189a329b29846a841: stopping tablet replica
I20240502 07:58:47.456792 9174 raft_consensus.cc:2229] T d41a2cb7b69e40d9a681384e5ad0a451 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:47.456727 5760 raft_consensus.cc:2932] T d41a2cb7b69e40d9a681384e5ad0a451 P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Committing config change with OpId 1.281: config changed from index 280 to 281, VOTER 91e82003a656442189a329b29846a841 (127.3.139.65) evicted. New config: { opid_index: 281 OBSOLETE_local: false peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.456912 9174 raft_consensus.cc:2258] T d41a2cb7b69e40d9a681384e5ad0a451 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:47.458619 9174 ts_tablet_manager.cc:1875] T d41a2cb7b69e40d9a681384e5ad0a451 P 91e82003a656442189a329b29846a841: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:58:47.461102 9174 ts_tablet_manager.cc:1888] T d41a2cb7b69e40d9a681384e5ad0a451 P 91e82003a656442189a329b29846a841: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.280
I20240502 07:58:47.461174 9174 log.cc:1196] T d41a2cb7b69e40d9a681384e5ad0a451 P 91e82003a656442189a329b29846a841: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal/wals/d41a2cb7b69e40d9a681384e5ad0a451
I20240502 07:58:47.461623 5473 catalog_manager.cc:4830] TS 91e82003a656442189a329b29846a841 (127.3.139.65:43429): tablet d41a2cb7b69e40d9a681384e5ad0a451 (table rebalance_test_table_1 [id=6f5d44f6fbe544099b33ad94e35cee6e]) successfully deleted
I20240502 07:58:47.511449 8251 raft_consensus.cc:1050] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841: attempting to promote NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc to VOTER
I20240502 07:58:47.511936 8251 consensus_queue.cc:237] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 501, Committed index: 501, Last appended: 1.501, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 502 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } }
I20240502 07:58:47.513478 8322 raft_consensus.cc:1261] T 96dcb06388374514951f24393f288eca P 389b75652ad34e55b571eb61b5d9c7bc [term 1 LEARNER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 501. Preceding OpId from leader: term: 1 index: 502. (index mismatch)
I20240502 07:58:47.513499 6151 raft_consensus.cc:1261] T 96dcb06388374514951f24393f288eca P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 501. Preceding OpId from leader: term: 1 index: 502. (index mismatch)
I20240502 07:58:47.513557 7044 raft_consensus.cc:1261] T 96dcb06388374514951f24393f288eca P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 501. Preceding OpId from leader: term: 1 index: 502. (index mismatch)
I20240502 07:58:47.513931 9101 consensus_queue.cc:1035] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 502, Last known committed idx: 501, Time since last communication: 0.000s
I20240502 07:58:47.514122 8251 consensus_queue.cc:1035] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 502, Last known committed idx: 501, Time since last communication: 0.000s
I20240502 07:58:47.514302 8787 consensus_queue.cc:1035] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 502, Last known committed idx: 501, Time since last communication: 0.000s
I20240502 07:58:47.514349 5891 raft_consensus.cc:1261] T 96dcb06388374514951f24393f288eca P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 501. Preceding OpId from leader: term: 1 index: 502. (index mismatch)
I20240502 07:58:47.514586 5760 raft_consensus.cc:1261] T 96dcb06388374514951f24393f288eca P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 501. Preceding OpId from leader: term: 1 index: 502. (index mismatch)
I20240502 07:58:47.514688 8787 consensus_queue.cc:1035] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 502, Last known committed idx: 501, Time since last communication: 0.000s
I20240502 07:58:47.514969 8787 consensus_queue.cc:1035] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 502, Last known committed idx: 501, Time since last communication: 0.000s
I20240502 07:58:47.516705 9101 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [term 1 LEADER]: Committing config change with OpId 1.502: config changed from index 501 to 502, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 502 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.517056 8322 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P 389b75652ad34e55b571eb61b5d9c7bc [term 1 FOLLOWER]: Committing config change with OpId 1.502: config changed from index 501 to 502, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 502 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.517925 5891 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Committing config change with OpId 1.502: config changed from index 501 to 502, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 502 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.518837 7044 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Committing config change with OpId 1.502: config changed from index 501 to 502, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 502 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.519371 5760 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Committing config change with OpId 1.502: config changed from index 501 to 502, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 502 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.519582 5482 catalog_manager.cc:5484] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 reported cstate change: config changed from index 501 to 502, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "91e82003a656442189a329b29846a841" committed_config { opid_index: 502 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:58:47.521159 6151 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.502: config changed from index 501 to 502, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 502 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } attrs { replace: true } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.524022 5627 consensus_queue.cc:237] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 502, Committed index: 502, Last appended: 1.502, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 503 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } }
I20240502 07:58:47.525565 5760 raft_consensus.cc:1261] T 96dcb06388374514951f24393f288eca P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 502. Preceding OpId from leader: term: 1 index: 503. (index mismatch)
I20240502 07:58:47.525704 5891 raft_consensus.cc:1261] T 96dcb06388374514951f24393f288eca P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 502. Preceding OpId from leader: term: 1 index: 503. (index mismatch)
I20240502 07:58:47.526266 8251 consensus_queue.cc:1035] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 503, Last known committed idx: 502, Time since last communication: 0.000s
I20240502 07:58:47.526576 8251 consensus_queue.cc:1035] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 503, Last known committed idx: 502, Time since last communication: 0.000s
I20240502 07:58:47.526908 7044 raft_consensus.cc:1261] T 96dcb06388374514951f24393f288eca P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 502. Preceding OpId from leader: term: 1 index: 503. (index mismatch)
I20240502 07:58:47.527253 8322 raft_consensus.cc:1261] T 96dcb06388374514951f24393f288eca P 389b75652ad34e55b571eb61b5d9c7bc [term 1 FOLLOWER]: Refusing update from remote peer 91e82003a656442189a329b29846a841: Log matching property violated. Preceding OpId in replica: term: 1 index: 502. Preceding OpId from leader: term: 1 index: 503. (index mismatch)
I20240502 07:58:47.527283 8787 consensus_queue.cc:1035] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 503, Last known committed idx: 502, Time since last communication: 0.000s
I20240502 07:58:47.527649 9101 consensus_queue.cc:1035] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 503, Last known committed idx: 502, Time since last communication: 0.000s
I20240502 07:58:47.528684 8787 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 [term 1 LEADER]: Committing config change with OpId 1.503: config changed from index 502 to 503, VOTER e020e7ffe8ce423a8bdefec667161217 (127.3.139.69) evicted. New config: { opid_index: 503 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.529587 8322 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P 389b75652ad34e55b571eb61b5d9c7bc [term 1 FOLLOWER]: Committing config change with OpId 1.503: config changed from index 502 to 503, VOTER e020e7ffe8ce423a8bdefec667161217 (127.3.139.69) evicted. New config: { opid_index: 503 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.530211 5473 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 96dcb06388374514951f24393f288eca with cas_config_opid_index 502: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:58:47.530349 5760 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P eda55a8566c447cfa749457825b70688 [term 1 FOLLOWER]: Committing config change with OpId 1.503: config changed from index 502 to 503, VOTER e020e7ffe8ce423a8bdefec667161217 (127.3.139.69) evicted. New config: { opid_index: 503 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.530898 5486 catalog_manager.cc:5484] T 96dcb06388374514951f24393f288eca P 91e82003a656442189a329b29846a841 reported cstate change: config changed from index 502 to 503, VOTER e020e7ffe8ce423a8bdefec667161217 (127.3.139.69) evicted. New cstate: current_term: 1 leader_uuid: "91e82003a656442189a329b29846a841" committed_config { opid_index: 503 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:58:47.532644 7044 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P 27e3fafad7a24bcda362d2197be87017 [term 1 FOLLOWER]: Committing config change with OpId 1.503: config changed from index 502 to 503, VOTER e020e7ffe8ce423a8bdefec667161217 (127.3.139.69) evicted. New config: { opid_index: 503 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.534478 6132 tablet_service.cc:1514] Processing DeleteTablet for tablet 96dcb06388374514951f24393f288eca with delete_type TABLET_DATA_TOMBSTONED (TS e020e7ffe8ce423a8bdefec667161217 not found in new config with opid_index 503) from {username='slave'} at 127.0.0.1:46442
I20240502 07:58:47.535446 5891 raft_consensus.cc:2932] T 96dcb06388374514951f24393f288eca P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Committing config change with OpId 1.503: config changed from index 502 to 503, VOTER e020e7ffe8ce423a8bdefec667161217 (127.3.139.69) evicted. New config: { opid_index: 503 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "27e3fafad7a24bcda362d2197be87017" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 35341 } attrs { promote: false } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { promote: false } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.539242 9202 tablet_replica.cc:330] T 96dcb06388374514951f24393f288eca P e020e7ffe8ce423a8bdefec667161217: stopping tablet replica
I20240502 07:58:47.539363 9202 raft_consensus.cc:2229] T 96dcb06388374514951f24393f288eca P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:47.539520 9202 raft_consensus.cc:2258] T 96dcb06388374514951f24393f288eca P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:47.540587 9202 ts_tablet_manager.cc:1875] T 96dcb06388374514951f24393f288eca P e020e7ffe8ce423a8bdefec667161217: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:58:47.542165 9202 ts_tablet_manager.cc:1888] T 96dcb06388374514951f24393f288eca P e020e7ffe8ce423a8bdefec667161217: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.502
I20240502 07:58:47.542235 9202 log.cc:1196] T 96dcb06388374514951f24393f288eca P e020e7ffe8ce423a8bdefec667161217: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/wal/wals/96dcb06388374514951f24393f288eca
I20240502 07:58:47.542742 5474 catalog_manager.cc:4830] TS e020e7ffe8ce423a8bdefec667161217 (127.3.139.69:39399): tablet 96dcb06388374514951f24393f288eca (table rebalance_test_table_3 [id=d0a0bf64867c4915ad797ab2bcd7101f]) successfully deleted
I20240502 07:58:47.580996 8867 raft_consensus.cc:1050] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688: attempting to promote NON_VOTER 389b75652ad34e55b571eb61b5d9c7bc to VOTER
I20240502 07:58:47.581364 8867 consensus_queue.cc:237] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 293, Committed index: 293, Last appended: 1.293, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 294 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } }
I20240502 07:58:47.582739 8322 raft_consensus.cc:1261] T 635b0a1a0fd64eeeac70025014c9ab81 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 LEARNER]: Refusing update from remote peer eda55a8566c447cfa749457825b70688: Log matching property violated. Preceding OpId in replica: term: 1 index: 293. Preceding OpId from leader: term: 1 index: 294. (index mismatch)
I20240502 07:58:47.582976 6281 raft_consensus.cc:1261] T 635b0a1a0fd64eeeac70025014c9ab81 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Refusing update from remote peer eda55a8566c447cfa749457825b70688: Log matching property violated. Preceding OpId in replica: term: 1 index: 293. Preceding OpId from leader: term: 1 index: 294. (index mismatch)
I20240502 07:58:47.583119 5627 raft_consensus.cc:1261] T 635b0a1a0fd64eeeac70025014c9ab81 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer eda55a8566c447cfa749457825b70688: Log matching property violated. Preceding OpId in replica: term: 1 index: 293. Preceding OpId from leader: term: 1 index: 294. (index mismatch)
I20240502 07:58:47.583173 9208 consensus_queue.cc:1035] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688 [LEADER]: Connected to new peer: Peer: permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 294, Last known committed idx: 293, Time since last communication: 0.000s
I20240502 07:58:47.583295 5891 raft_consensus.cc:1261] T 635b0a1a0fd64eeeac70025014c9ab81 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Refusing update from remote peer eda55a8566c447cfa749457825b70688: Log matching property violated. Preceding OpId in replica: term: 1 index: 293. Preceding OpId from leader: term: 1 index: 294. (index mismatch)
I20240502 07:58:47.583503 6151 raft_consensus.cc:1261] T 635b0a1a0fd64eeeac70025014c9ab81 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer eda55a8566c447cfa749457825b70688: Log matching property violated. Preceding OpId in replica: term: 1 index: 293. Preceding OpId from leader: term: 1 index: 294. (index mismatch)
I20240502 07:58:47.583621 9206 consensus_queue.cc:1035] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 294, Last known committed idx: 293, Time since last communication: 0.000s
I20240502 07:58:47.583896 9203 consensus_queue.cc:1035] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 294, Last known committed idx: 293, Time since last communication: 0.000s
I20240502 07:58:47.584075 9207 consensus_queue.cc:1035] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 294, Last known committed idx: 293, Time since last communication: 0.000s
I20240502 07:58:47.584322 9205 consensus_queue.cc:1035] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 294, Last known committed idx: 293, Time since last communication: 0.000s
I20240502 07:58:47.586056 9206 raft_consensus.cc:2932] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688 [term 1 LEADER]: Committing config change with OpId 1.294: config changed from index 293 to 294, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 294 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.586635 6151 raft_consensus.cc:2932] T 635b0a1a0fd64eeeac70025014c9ab81 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.294: config changed from index 293 to 294, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 294 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.586828 6281 raft_consensus.cc:2932] T 635b0a1a0fd64eeeac70025014c9ab81 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Committing config change with OpId 1.294: config changed from index 293 to 294, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 294 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.588181 8322 raft_consensus.cc:2932] T 635b0a1a0fd64eeeac70025014c9ab81 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 FOLLOWER]: Committing config change with OpId 1.294: config changed from index 293 to 294, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 294 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.588737 5486 catalog_manager.cc:5484] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688 reported cstate change: config changed from index 293 to 294, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "eda55a8566c447cfa749457825b70688" committed_config { opid_index: 294 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:58:47.589802 5627 raft_consensus.cc:2932] T 635b0a1a0fd64eeeac70025014c9ab81 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Committing config change with OpId 1.294: config changed from index 293 to 294, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 294 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.590559 5891 raft_consensus.cc:2932] T 635b0a1a0fd64eeeac70025014c9ab81 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Committing config change with OpId 1.294: config changed from index 293 to 294, 389b75652ad34e55b571eb61b5d9c7bc (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 294 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "89bf5c0a6b2a4d949851089446f17b96" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 37141 } attrs { replace: true } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.592967 5760 consensus_queue.cc:237] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 294, Committed index: 294, Last appended: 1.294, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 295 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } }
I20240502 07:58:47.594380 8322 raft_consensus.cc:1261] T 635b0a1a0fd64eeeac70025014c9ab81 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 FOLLOWER]: Refusing update from remote peer eda55a8566c447cfa749457825b70688: Log matching property violated. Preceding OpId in replica: term: 1 index: 294. Preceding OpId from leader: term: 1 index: 295. (index mismatch)
I20240502 07:58:47.594636 5627 raft_consensus.cc:1261] T 635b0a1a0fd64eeeac70025014c9ab81 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Refusing update from remote peer eda55a8566c447cfa749457825b70688: Log matching property violated. Preceding OpId in replica: term: 1 index: 294. Preceding OpId from leader: term: 1 index: 295. (index mismatch)
I20240502 07:58:47.594815 9205 consensus_queue.cc:1035] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688 [LEADER]: Connected to new peer: Peer: permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 295, Last known committed idx: 294, Time since last communication: 0.000s
I20240502 07:58:47.595114 9205 consensus_queue.cc:1035] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 295, Last known committed idx: 294, Time since last communication: 0.000s
I20240502 07:58:47.595770 6151 raft_consensus.cc:1261] T 635b0a1a0fd64eeeac70025014c9ab81 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Refusing update from remote peer eda55a8566c447cfa749457825b70688: Log matching property violated. Preceding OpId in replica: term: 1 index: 294. Preceding OpId from leader: term: 1 index: 295. (index mismatch)
I20240502 07:58:47.596089 9208 consensus_queue.cc:1035] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 295, Last known committed idx: 294, Time since last communication: 0.000s
I20240502 07:58:47.596155 6281 raft_consensus.cc:1261] T 635b0a1a0fd64eeeac70025014c9ab81 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Refusing update from remote peer eda55a8566c447cfa749457825b70688: Log matching property violated. Preceding OpId in replica: term: 1 index: 294. Preceding OpId from leader: term: 1 index: 295. (index mismatch)
I20240502 07:58:47.596449 9207 consensus_queue.cc:1035] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 295, Last known committed idx: 294, Time since last communication: 0.000s
I20240502 07:58:47.596690 9205 raft_consensus.cc:2932] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688 [term 1 LEADER]: Committing config change with OpId 1.295: config changed from index 294 to 295, VOTER 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67) evicted. New config: { opid_index: 295 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.597294 6281 raft_consensus.cc:2932] T 635b0a1a0fd64eeeac70025014c9ab81 P d58f6d1ff8cf472f8a9edf3700e64bfb [term 1 FOLLOWER]: Committing config change with OpId 1.295: config changed from index 294 to 295, VOTER 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67) evicted. New config: { opid_index: 295 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.597911 5473 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 635b0a1a0fd64eeeac70025014c9ab81 with cas_config_opid_index 294: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:58:47.598340 8322 raft_consensus.cc:2932] T 635b0a1a0fd64eeeac70025014c9ab81 P 389b75652ad34e55b571eb61b5d9c7bc [term 1 FOLLOWER]: Committing config change with OpId 1.295: config changed from index 294 to 295, VOTER 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67) evicted. New config: { opid_index: 295 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.598428 5627 raft_consensus.cc:2932] T 635b0a1a0fd64eeeac70025014c9ab81 P 91e82003a656442189a329b29846a841 [term 1 FOLLOWER]: Committing config change with OpId 1.295: config changed from index 294 to 295, VOTER 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67) evicted. New config: { opid_index: 295 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.599018 5482 catalog_manager.cc:5484] T 635b0a1a0fd64eeeac70025014c9ab81 P eda55a8566c447cfa749457825b70688 reported cstate change: config changed from index 294 to 295, VOTER 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67) evicted. New cstate: current_term: 1 leader_uuid: "eda55a8566c447cfa749457825b70688" committed_config { opid_index: 295 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:58:47.599927 6151 raft_consensus.cc:2932] T 635b0a1a0fd64eeeac70025014c9ab81 P e020e7ffe8ce423a8bdefec667161217 [term 1 FOLLOWER]: Committing config change with OpId 1.295: config changed from index 294 to 295, VOTER 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67) evicted. New config: { opid_index: 295 OBSOLETE_local: false peers { permanent_uuid: "eda55a8566c447cfa749457825b70688" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 39925 } } peers { permanent_uuid: "91e82003a656442189a329b29846a841" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 43429 } } peers { permanent_uuid: "e020e7ffe8ce423a8bdefec667161217" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 39399 } } peers { permanent_uuid: "d58f6d1ff8cf472f8a9edf3700e64bfb" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 41047 } } peers { permanent_uuid: "389b75652ad34e55b571eb61b5d9c7bc" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36507 } attrs { promote: false } } }
I20240502 07:58:47.601922 5872 tablet_service.cc:1514] Processing DeleteTablet for tablet 635b0a1a0fd64eeeac70025014c9ab81 with delete_type TABLET_DATA_TOMBSTONED (TS 89bf5c0a6b2a4d949851089446f17b96 not found in new config with opid_index 295) from {username='slave'} at 127.0.0.1:42126
I20240502 07:58:47.602269 9214 tablet_replica.cc:330] T 635b0a1a0fd64eeeac70025014c9ab81 P 89bf5c0a6b2a4d949851089446f17b96: stopping tablet replica
I20240502 07:58:47.602432 9214 raft_consensus.cc:2229] T 635b0a1a0fd64eeeac70025014c9ab81 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:47.602721 9214 raft_consensus.cc:2258] T 635b0a1a0fd64eeeac70025014c9ab81 P 89bf5c0a6b2a4d949851089446f17b96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:47.603958 9214 ts_tablet_manager.cc:1875] T 635b0a1a0fd64eeeac70025014c9ab81 P 89bf5c0a6b2a4d949851089446f17b96: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:58:47.605508 9214 ts_tablet_manager.cc:1888] T 635b0a1a0fd64eeeac70025014c9ab81 P 89bf5c0a6b2a4d949851089446f17b96: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.294
I20240502 07:58:47.605569 9214 log.cc:1196] T 635b0a1a0fd64eeeac70025014c9ab81 P 89bf5c0a6b2a4d949851089446f17b96: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal/wals/635b0a1a0fd64eeeac70025014c9ab81
I20240502 07:58:47.606001 5473 catalog_manager.cc:4830] TS 89bf5c0a6b2a4d949851089446f17b96 (127.3.139.67:37141): tablet 635b0a1a0fd64eeeac70025014c9ab81 (table rebalance_test_table_2 [id=28039a31ea894d23af67c1f1da387f21]) successfully deleted
I20240502 07:58:47.752115 8307 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:47.754395 7028 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:47.763118 5608 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:47.766978 5872 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:47.772318 6263 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:58:47.774343 5740 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:47.774751 6002 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:58:47.777081 6132 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:47.880729 8307 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:47.883481 5872 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:47.885915 7028 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:47.888643 5608 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:47.894244 6263 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:58:47.898308 5740 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:47.900768 6002 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:58:47.920615 6132 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:48.006302 7028 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:48.006947 5608 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:48.011930 5872 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:48.013794 6263 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:58:48.019501 5740 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:48.019783 8307 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:48.020066 6002 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:58:48.027323 6132 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
c5d1643b3f5747ccb4514a400dcc82b4 | 127.3.139.126:34225 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
allow_unsafe_replication_factor | true | runtime,unsafe | all 1 server(s) checked
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.3.139.84:41319 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
27e3fafad7a24bcda362d2197be87017 | 127.3.139.71:35341 | HEALTHY | <none> | 0 | 0
389b75652ad34e55b571eb61b5d9c7bc | 127.3.139.72:36507 | HEALTHY | <none> | 0 | 0
89bf5c0a6b2a4d949851089446f17b96 | 127.3.139.67:37141 | HEALTHY | <none> | 1 | 0
91e82003a656442189a329b29846a841 | 127.3.139.65:43429 | HEALTHY | <none> | 3 | 0
d58f6d1ff8cf472f8a9edf3700e64bfb | 127.3.139.70:41047 | HEALTHY | <none> | 4 | 0
d5bc7863cd294f77a932c4a40f98b79d | 127.3.139.68:44287 | HEALTHY | <none> | 4 | 0
e020e7ffe8ce423a8bdefec667161217 | 127.3.139.69:39399 | HEALTHY | <none> | 1 | 0
eda55a8566c447cfa749457825b70688 | 127.3.139.66:39925 | HEALTHY | <none> | 3 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 8
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_server_key_size | 768 | experimental | all 8 server(s) checked
local_ip_for_outbound_sockets | 127.3.139.65 | experimental | 127.3.139.65:43429
local_ip_for_outbound_sockets | 127.3.139.66 | experimental | 127.3.139.66:39925
local_ip_for_outbound_sockets | 127.3.139.67 | experimental | 127.3.139.67:37141
local_ip_for_outbound_sockets | 127.3.139.68 | experimental | 127.3.139.68:44287
local_ip_for_outbound_sockets | 127.3.139.69 | experimental | 127.3.139.69:39399
local_ip_for_outbound_sockets | 127.3.139.70 | experimental | 127.3.139.70:41047
local_ip_for_outbound_sockets | 127.3.139.71 | experimental | 127.3.139.71:35341
local_ip_for_outbound_sockets | 127.3.139.72 | experimental | 127.3.139.72:36507
never_fsync | true | unsafe,advanced | all 8 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 8 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 8 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 8 server(s) checked
server_dump_info_format | pb | hidden | all 8 server(s) checked
server_dump_info_path | /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden | 127.3.139.65:43429
server_dump_info_path | /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.3.139.66:39925
server_dump_info_path | /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.3.139.67:37141
server_dump_info_path | /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.3.139.68:44287
server_dump_info_path | /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.3.139.69:39399
server_dump_info_path | /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.3.139.70:41047
server_dump_info_path | /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/data/info.pb | hidden | 127.3.139.71:35341
server_dump_info_path | /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/data/info.pb | hidden | 127.3.139.72:36507
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.3.139.84:41319 | all 8 server(s) checked
time_source | builtin | all 8 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 9 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
------------------------+----+---------+---------------+---------+------------+------------------+-------------
TestTable | 5 | HEALTHY | 1 | 1 | 0 | 0 | 0
rebalance_test_table_0 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_1 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_2 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_3 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_4 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 10
First Quartile | 10
Median | 10
Third Quartile | 10
Maximum | 10
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 8
Tables | 6
Tablets | 16
Replicas | 80
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20240502 07:58:48.067732 3629 log_verifier.cc:126] Checking tablet 07d53332bdd34233b3653756770dc9cf
I20240502 07:58:48.171517 3629 log_verifier.cc:177] Verified matching terms for 203 ops in tablet 07d53332bdd34233b3653756770dc9cf
I20240502 07:58:48.171653 3629 log_verifier.cc:126] Checking tablet 18a448dbe2af493abefbc728a8f85186
I20240502 07:58:48.301476 3629 log_verifier.cc:177] Verified matching terms for 314 ops in tablet 18a448dbe2af493abefbc728a8f85186
I20240502 07:58:48.301697 3629 log_verifier.cc:126] Checking tablet 2a3919f144ed4a27b03e7c4a251499d4
I20240502 07:58:48.466476 3629 log_verifier.cc:177] Verified matching terms for 426 ops in tablet 2a3919f144ed4a27b03e7c4a251499d4
I20240502 07:58:48.466701 3629 log_verifier.cc:126] Checking tablet 5b87554a301044caac956bc5c7f137e8
I20240502 07:58:48.593327 3629 log_verifier.cc:177] Verified matching terms for 296 ops in tablet 5b87554a301044caac956bc5c7f137e8
I20240502 07:58:48.593508 3629 log_verifier.cc:126] Checking tablet 5c8c7437c85943ba9113595c17bf6136
I20240502 07:58:48.715669 3629 log_verifier.cc:177] Verified matching terms for 289 ops in tablet 5c8c7437c85943ba9113595c17bf6136
I20240502 07:58:48.715826 3629 log_verifier.cc:126] Checking tablet 635b0a1a0fd64eeeac70025014c9ab81
I20240502 07:58:48.842070 3629 log_verifier.cc:177] Verified matching terms for 295 ops in tablet 635b0a1a0fd64eeeac70025014c9ab81
I20240502 07:58:48.842270 3629 log_verifier.cc:126] Checking tablet 7001a52697f1417bb84a0e3ed673ed0c
I20240502 07:58:48.970749 3629 log_verifier.cc:177] Verified matching terms for 312 ops in tablet 7001a52697f1417bb84a0e3ed673ed0c
I20240502 07:58:48.970947 3629 log_verifier.cc:126] Checking tablet 7f3d5d24bc164883adb1a11552500ce6
I20240502 07:58:49.050338 3629 log_verifier.cc:177] Verified matching terms for 162 ops in tablet 7f3d5d24bc164883adb1a11552500ce6
I20240502 07:58:49.050467 3629 log_verifier.cc:126] Checking tablet 96dcb06388374514951f24393f288eca
I20240502 07:58:49.244057 3629 log_verifier.cc:177] Verified matching terms for 503 ops in tablet 96dcb06388374514951f24393f288eca
I20240502 07:58:49.244328 3629 log_verifier.cc:126] Checking tablet acf2ff61e8ff450ab7fca71fb4191dd6
I20240502 07:58:49.377915 3629 log_verifier.cc:177] Verified matching terms for 319 ops in tablet acf2ff61e8ff450ab7fca71fb4191dd6
I20240502 07:58:49.378098 3629 log_verifier.cc:126] Checking tablet bc41794fb8c545149694aa84dde1cdb0
I20240502 07:58:49.558566 3629 log_verifier.cc:177] Verified matching terms for 451 ops in tablet bc41794fb8c545149694aa84dde1cdb0
I20240502 07:58:49.558908 3629 log_verifier.cc:126] Checking tablet cd070443095547d1bf82a6c4a7666b86
I20240502 07:58:49.586544 3629 log_verifier.cc:177] Verified matching terms for 7 ops in tablet cd070443095547d1bf82a6c4a7666b86
I20240502 07:58:49.586611 3629 log_verifier.cc:126] Checking tablet d14545ba1e474960909a1b140b6a315c
I20240502 07:58:49.716171 3629 log_verifier.cc:177] Verified matching terms for 293 ops in tablet d14545ba1e474960909a1b140b6a315c
I20240502 07:58:49.716336 3629 log_verifier.cc:126] Checking tablet d41a2cb7b69e40d9a681384e5ad0a451
I20240502 07:58:49.839504 3629 log_verifier.cc:177] Verified matching terms for 281 ops in tablet d41a2cb7b69e40d9a681384e5ad0a451
I20240502 07:58:49.839669 3629 log_verifier.cc:126] Checking tablet eede21c622d8435ba1efa473ef97b1d1
I20240502 07:58:49.925595 3629 log_verifier.cc:177] Verified matching terms for 169 ops in tablet eede21c622d8435ba1efa473ef97b1d1
I20240502 07:58:49.925748 3629 log_verifier.cc:126] Checking tablet f1a11f8d412949218b8af5de7c63505f
I20240502 07:58:50.045166 3629 log_verifier.cc:177] Verified matching terms for 294 ops in tablet f1a11f8d412949218b8af5de7c63505f
I20240502 07:58:50.046572 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 5548
I20240502 07:58:50.054458 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 5679
I20240502 07:58:50.060662 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 5810
I20240502 07:58:50.066175 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 5941
I20240502 07:58:50.071583 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 6072
I20240502 07:58:50.076741 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 6203
I20240502 07:58:50.081881 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 6966
I20240502 07:58:50.086526 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 8253
I20240502 07:58:50.090819 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 5457
2024-05-02T07:58:50Z chronyd exiting
[ OK ] RebalanceParamTest.Rebalance/7 (34187 ms)
[----------] 1 test from RebalanceParamTest (34187 ms total)
[----------] 1 test from TserverAddedDuringRebalancingTest
[ RUN ] TserverAddedDuringRebalancingTest.TserverStarts/1
I20240502 07:58:50.171590 3629 test_util.cc:268] Using random seed: -113737534
I20240502 07:58:50.172318 3629 ts_itest-base.cc:112] Starting cluster with:
I20240502 07:58:50.172355 3629 ts_itest-base.cc:113] --------------
I20240502 07:58:50.172365 3629 ts_itest-base.cc:114] 8 tablet servers
I20240502 07:58:50.172374 3629 ts_itest-base.cc:115] 3 replicas per TS
I20240502 07:58:50.172386 3629 ts_itest-base.cc:116] --------------
2024-05-02T07:58:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-02T07:58:50Z Disabled control of system clock
I20240502 07:58:50.180284 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.139.126:46479
--webserver_interface=127.3.139.126
--webserver_port=0
--builtin_ntp_servers=127.3.139.84:39259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.139.126:46479
--tserver_unresponsive_timeout_ms=3000
--enable_range_replica_placement=false
--raft_prepare_replacement_before_eviction=true
--enable_range_replica_placement=false with env {}
W20240502 07:58:50.277444 9342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:50.277628 9342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:50.277664 9342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:50.280551 9342 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:58:50.280611 9342 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240502 07:58:50.280632 9342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:50.280647 9342 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240502 07:58:50.280673 9342 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240502 07:58:50.284117 9342 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:39259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.3.139.126:46479
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.139.126:46479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.3.139.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.9342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:50.284830 9342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:50.285786 9342 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:50.292558 9347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:50.292963 9348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:50.293313 9350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:50.293751 9342 server_base.cc:1034] running on GCE node
I20240502 07:58:50.294165 9342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:50.294962 9342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:50.296103 9342 hybrid_clock.cc:648] HybridClock initialized: now 1714636730296091 us; error 27 us; skew 500 ppm
I20240502 07:58:50.302187 9342 webserver.cc:460] Webserver started at http://127.3.139.126:40295/ using document root <none> and password file <none>
I20240502 07:58:50.302678 9342 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:50.302727 9342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:50.302874 9342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:50.304388 9342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "bcbc6b7ce1a14c38a0a71655f6b9fd9c"
format_stamp: "Formatted at 2024-05-02 07:58:50 on dist-test-slave-twcr"
I20240502 07:58:50.304800 9342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "bcbc6b7ce1a14c38a0a71655f6b9fd9c"
format_stamp: "Formatted at 2024-05-02 07:58:50 on dist-test-slave-twcr"
I20240502 07:58:50.307760 9342 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20240502 07:58:50.309926 9356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:50.310835 9342 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20240502 07:58:50.310938 9342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal
uuid: "bcbc6b7ce1a14c38a0a71655f6b9fd9c"
format_stamp: "Formatted at 2024-05-02 07:58:50 on dist-test-slave-twcr"
I20240502 07:58:50.311024 9342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:50.333145 9342 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:50.333675 9342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:50.333810 9342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:50.339903 9342 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.126:46479
I20240502 07:58:50.339924 9408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.126:46479 every 8 connection(s)
I20240502 07:58:50.340768 9342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240502 07:58:50.343336 9409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:50.344317 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 9342
I20240502 07:58:50.344419 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240502 07:58:50.348697 9409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcbc6b7ce1a14c38a0a71655f6b9fd9c: Bootstrap starting.
I20240502 07:58:50.350847 9409 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bcbc6b7ce1a14c38a0a71655f6b9fd9c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:50.351658 9409 log.cc:824] T 00000000000000000000000000000000 P bcbc6b7ce1a14c38a0a71655f6b9fd9c: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:50.353037 9409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcbc6b7ce1a14c38a0a71655f6b9fd9c: No bootstrap required, opened a new log
I20240502 07:58:50.355270 9409 raft_consensus.cc:348] T 00000000000000000000000000000000 P bcbc6b7ce1a14c38a0a71655f6b9fd9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcbc6b7ce1a14c38a0a71655f6b9fd9c" member_type: VOTER last_known_addr { host: "127.3.139.126" port: 46479 } }
I20240502 07:58:50.355435 9409 raft_consensus.cc:374] T 00000000000000000000000000000000 P bcbc6b7ce1a14c38a0a71655f6b9fd9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:50.355490 9409 raft_consensus.cc:729] T 00000000000000000000000000000000 P bcbc6b7ce1a14c38a0a71655f6b9fd9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcbc6b7ce1a14c38a0a71655f6b9fd9c, State: Initialized, Role: FOLLOWER
I20240502 07:58:50.355857 9409 consensus_queue.cc:260] T 00000000000000000000000000000000 P bcbc6b7ce1a14c38a0a71655f6b9fd9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcbc6b7ce1a14c38a0a71655f6b9fd9c" member_type: VOTER last_known_addr { host: "127.3.139.126" port: 46479 } }
I20240502 07:58:50.355968 9409 raft_consensus.cc:388] T 00000000000000000000000000000000 P bcbc6b7ce1a14c38a0a71655f6b9fd9c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 07:58:50.356022 9409 raft_consensus.cc:482] T 00000000000000000000000000000000 P bcbc6b7ce1a14c38a0a71655f6b9fd9c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 07:58:50.356117 9409 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bcbc6b7ce1a14c38a0a71655f6b9fd9c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:50.356724 9409 raft_consensus.cc:504] T 00000000000000000000000000000000 P bcbc6b7ce1a14c38a0a71655f6b9fd9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcbc6b7ce1a14c38a0a71655f6b9fd9c" member_type: VOTER last_known_addr { host: "127.3.139.126" port: 46479 } }
I20240502 07:58:50.356997 9409 leader_election.cc:302] T 00000000000000000000000000000000 P bcbc6b7ce1a14c38a0a71655f6b9fd9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bcbc6b7ce1a14c38a0a71655f6b9fd9c; no voters:
I20240502 07:58:50.357211 9409 leader_election.cc:288] T 00000000000000000000000000000000 P bcbc6b7ce1a14c38a0a71655f6b9fd9c [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 07:58:50.357295 9414 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bcbc6b7ce1a14c38a0a71655f6b9fd9c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:50.357479 9414 raft_consensus.cc:686] T 00000000000000000000000000000000 P bcbc6b7ce1a14c38a0a71655f6b9fd9c [term 1 LEADER]: Becoming Leader. State: Replica: bcbc6b7ce1a14c38a0a71655f6b9fd9c, State: Running, Role: LEADER
I20240502 07:58:50.357930 9414 consensus_queue.cc:237] T 00000000000000000000000000000000 P bcbc6b7ce1a14c38a0a71655f6b9fd9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcbc6b7ce1a14c38a0a71655f6b9fd9c" member_type: VOTER last_known_addr { host: "127.3.139.126" port: 46479 } }
I20240502 07:58:50.358175 9409 sys_catalog.cc:564] T 00000000000000000000000000000000 P bcbc6b7ce1a14c38a0a71655f6b9fd9c [sys.catalog]: configured and running, proceeding with master startup.
I20240502 07:58:50.359391 9415 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcbc6b7ce1a14c38a0a71655f6b9fd9c [sys.catalog]: SysCatalogTable state changed. Reason: New leader bcbc6b7ce1a14c38a0a71655f6b9fd9c. Latest consensus state: current_term: 1 leader_uuid: "bcbc6b7ce1a14c38a0a71655f6b9fd9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcbc6b7ce1a14c38a0a71655f6b9fd9c" member_type: VOTER last_known_addr { host: "127.3.139.126" port: 46479 } } }
I20240502 07:58:50.359401 9416 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcbc6b7ce1a14c38a0a71655f6b9fd9c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bcbc6b7ce1a14c38a0a71655f6b9fd9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcbc6b7ce1a14c38a0a71655f6b9fd9c" member_type: VOTER last_known_addr { host: "127.3.139.126" port: 46479 } } }
I20240502 07:58:50.359563 9415 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcbc6b7ce1a14c38a0a71655f6b9fd9c [sys.catalog]: This master's current role is: LEADER
I20240502 07:58:50.359602 9416 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcbc6b7ce1a14c38a0a71655f6b9fd9c [sys.catalog]: This master's current role is: LEADER
I20240502 07:58:50.360337 9423 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240502 07:58:50.362548 9423 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240502 07:58:50.367183 9423 catalog_manager.cc:1347] Generated new cluster ID: 05b3fd5ae56c4eb3a6905c41374e98f4
I20240502 07:58:50.367234 9423 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240502 07:58:50.373893 9423 catalog_manager.cc:1370] Generated new certificate authority record
I20240502 07:58:50.374615 9423 catalog_manager.cc:1504] Loading token signing keys...
I20240502 07:58:50.389990 9423 catalog_manager.cc:5857] T 00000000000000000000000000000000 P bcbc6b7ce1a14c38a0a71655f6b9fd9c: Generated new TSK 0
I20240502 07:58:50.390520 9423 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240502 07:58:50.393779 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.65:0
--local_ip_for_outbound_sockets=127.3.139.65
--webserver_interface=127.3.139.65
--webserver_port=0
--tserver_master_addrs=127.3.139.126:46479
--builtin_ntp_servers=127.3.139.84:39259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
W20240502 07:58:50.514230 9433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:50.514418 9433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:50.514468 9433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:50.517400 9433 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:58:50.517458 9433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:50.517541 9433 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.65
I20240502 07:58:50.520985 9433 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:39259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.3.139.65
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.3.139.126:46479
--never_fsync=true
--heap_profile_path=/tmp/kudu.9433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.65
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:50.521760 9433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:50.522781 9433 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:50.530679 9441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:50.531044 9439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:50.531404 9438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:50.531816 9433 server_base.cc:1034] running on GCE node
I20240502 07:58:50.532145 9433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:50.532613 9433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:50.533761 9433 hybrid_clock.cc:648] HybridClock initialized: now 1714636730533735 us; error 43 us; skew 500 ppm
I20240502 07:58:50.535439 9433 webserver.cc:460] Webserver started at http://127.3.139.65:46113/ using document root <none> and password file <none>
I20240502 07:58:50.535925 9433 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:50.535980 9433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:50.536136 9433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:50.537606 9433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "a201db721bda4dc3a4422dc8fa9bf72e"
format_stamp: "Formatted at 2024-05-02 07:58:50 on dist-test-slave-twcr"
I20240502 07:58:50.538028 9433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "a201db721bda4dc3a4422dc8fa9bf72e"
format_stamp: "Formatted at 2024-05-02 07:58:50 on dist-test-slave-twcr"
I20240502 07:58:50.541085 9433 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 07:58:50.542958 9447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:50.543859 9433 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:50.543963 9433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "a201db721bda4dc3a4422dc8fa9bf72e"
format_stamp: "Formatted at 2024-05-02 07:58:50 on dist-test-slave-twcr"
I20240502 07:58:50.544034 9433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:50.567711 9433 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:50.568305 9433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:50.568451 9433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:50.568969 9433 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:50.569851 9433 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:50.569898 9433 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:50.569949 9433 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:50.569975 9433 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:50.577705 9433 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.65:39551
I20240502 07:58:50.577731 9560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.65:39551 every 8 connection(s)
I20240502 07:58:50.578552 9433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240502 07:58:50.579391 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 9433
I20240502 07:58:50.579507 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240502 07:58:50.581596 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.66:0
--local_ip_for_outbound_sockets=127.3.139.66
--webserver_interface=127.3.139.66
--webserver_port=0
--tserver_master_addrs=127.3.139.126:46479
--builtin_ntp_servers=127.3.139.84:39259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240502 07:58:50.587563 9561 heartbeater.cc:347] Connected to a master server at 127.3.139.126:46479
I20240502 07:58:50.587765 9561 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:50.588238 9561 heartbeater.cc:511] Master 127.3.139.126:46479 requested a full tablet report, sending...
I20240502 07:58:50.589666 9373 ts_manager.cc:194] Registered new tserver with Master: a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551)
I20240502 07:58:50.591275 9373 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.65:33821
W20240502 07:58:50.681934 9564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:50.682125 9564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:50.682166 9564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:50.685011 9564 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:58:50.685062 9564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:50.685125 9564 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.66
I20240502 07:58:50.688452 9564 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:39259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.3.139.66
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.3.139.126:46479
--never_fsync=true
--heap_profile_path=/tmp/kudu.9564
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.66
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:50.689149 9564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:50.690093 9564 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:50.697407 9570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:50.697780 9569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:50.698164 9572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:50.698519 9564 server_base.cc:1034] running on GCE node
I20240502 07:58:50.698824 9564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:50.699314 9564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:50.700455 9564 hybrid_clock.cc:648] HybridClock initialized: now 1714636730700433 us; error 34 us; skew 500 ppm
I20240502 07:58:50.702031 9564 webserver.cc:460] Webserver started at http://127.3.139.66:34195/ using document root <none> and password file <none>
I20240502 07:58:50.702487 9564 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:50.702533 9564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:50.702692 9564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:50.704159 9564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "8337af1943564c0dad18b45a597b805a"
format_stamp: "Formatted at 2024-05-02 07:58:50 on dist-test-slave-twcr"
I20240502 07:58:50.705261 9564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "8337af1943564c0dad18b45a597b805a"
format_stamp: "Formatted at 2024-05-02 07:58:50 on dist-test-slave-twcr"
I20240502 07:58:50.708184 9564 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:50.709975 9578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:50.710839 9564 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:50.710923 9564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "8337af1943564c0dad18b45a597b805a"
format_stamp: "Formatted at 2024-05-02 07:58:50 on dist-test-slave-twcr"
I20240502 07:58:50.711000 9564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:50.725469 9564 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:50.725921 9564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:50.726058 9564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:50.726516 9564 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:50.727262 9564 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:50.727304 9564 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:50.727341 9564 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:50.727372 9564 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:50.734649 9564 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.66:46667
I20240502 07:58:50.734674 9691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.66:46667 every 8 connection(s)
I20240502 07:58:50.735419 9564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240502 07:58:50.736541 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 9564
I20240502 07:58:50.736653 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240502 07:58:50.738673 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.67:0
--local_ip_for_outbound_sockets=127.3.139.67
--webserver_interface=127.3.139.67
--webserver_port=0
--tserver_master_addrs=127.3.139.126:46479
--builtin_ntp_servers=127.3.139.84:39259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240502 07:58:50.744076 9692 heartbeater.cc:347] Connected to a master server at 127.3.139.126:46479
I20240502 07:58:50.744254 9692 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:50.744689 9692 heartbeater.cc:511] Master 127.3.139.126:46479 requested a full tablet report, sending...
I20240502 07:58:50.745529 9373 ts_manager.cc:194] Registered new tserver with Master: 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667)
I20240502 07:58:50.746043 9373 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.66:54519
W20240502 07:58:50.836788 9695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:50.836971 9695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:50.837029 9695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:50.839951 9695 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:58:50.840008 9695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:50.840101 9695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.67
I20240502 07:58:50.843565 9695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:39259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.3.139.67
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.3.139.126:46479
--never_fsync=true
--heap_profile_path=/tmp/kudu.9695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.67
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:50.844267 9695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:50.845229 9695 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:50.853331 9700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:50.853633 9701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:50.854091 9703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:50.854477 9695 server_base.cc:1034] running on GCE node
I20240502 07:58:50.854846 9695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:50.855376 9695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:50.856523 9695 hybrid_clock.cc:648] HybridClock initialized: now 1714636730856511 us; error 26 us; skew 500 ppm
I20240502 07:58:50.858311 9695 webserver.cc:460] Webserver started at http://127.3.139.67:38177/ using document root <none> and password file <none>
I20240502 07:58:50.858840 9695 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:50.858899 9695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:50.859102 9695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:50.860786 9695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "1472c64688914149a318c7b8850a3df3"
format_stamp: "Formatted at 2024-05-02 07:58:50 on dist-test-slave-twcr"
I20240502 07:58:50.861218 9695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "1472c64688914149a318c7b8850a3df3"
format_stamp: "Formatted at 2024-05-02 07:58:50 on dist-test-slave-twcr"
I20240502 07:58:50.864246 9695 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20240502 07:58:50.866221 9709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:50.867175 9695 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:50.867269 9695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "1472c64688914149a318c7b8850a3df3"
format_stamp: "Formatted at 2024-05-02 07:58:50 on dist-test-slave-twcr"
I20240502 07:58:50.867341 9695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:50.896318 9695 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:50.896868 9695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:50.897014 9695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:50.897480 9695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:50.898295 9695 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:50.898339 9695 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:50.898380 9695 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:50.898409 9695 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:50.905966 9695 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.67:43191
I20240502 07:58:50.905997 9822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.67:43191 every 8 connection(s)
I20240502 07:58:50.906788 9695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240502 07:58:50.913761 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 9695
I20240502 07:58:50.913862 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240502 07:58:50.914323 9823 heartbeater.cc:347] Connected to a master server at 127.3.139.126:46479
I20240502 07:58:50.914517 9823 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:50.915028 9823 heartbeater.cc:511] Master 127.3.139.126:46479 requested a full tablet report, sending...
I20240502 07:58:50.915841 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.68:0
--local_ip_for_outbound_sockets=127.3.139.68
--webserver_interface=127.3.139.68
--webserver_port=0
--tserver_master_addrs=127.3.139.126:46479
--builtin_ntp_servers=127.3.139.84:39259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240502 07:58:50.915997 9373 ts_manager.cc:194] Registered new tserver with Master: 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191)
I20240502 07:58:50.916502 9373 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.67:39209
W20240502 07:58:51.014014 9826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:51.014200 9826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:51.014245 9826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:51.017131 9826 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:58:51.017184 9826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:51.017247 9826 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.68
I20240502 07:58:51.020606 9826 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:39259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.68:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.3.139.68
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.3.139.126:46479
--never_fsync=true
--heap_profile_path=/tmp/kudu.9826
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.68
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:51.021238 9826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:51.022105 9826 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:51.029042 9834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:51.029385 9831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:51.029747 9832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:51.030138 9826 server_base.cc:1034] running on GCE node
I20240502 07:58:51.030426 9826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:51.030867 9826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:51.032004 9826 hybrid_clock.cc:648] HybridClock initialized: now 1714636731031979 us; error 37 us; skew 500 ppm
I20240502 07:58:51.033516 9826 webserver.cc:460] Webserver started at http://127.3.139.68:44707/ using document root <none> and password file <none>
I20240502 07:58:51.033965 9826 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:51.034014 9826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:51.034173 9826 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:51.035610 9826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "6cc5b9962ed24441bcba0c07b9f5438e"
format_stamp: "Formatted at 2024-05-02 07:58:51 on dist-test-slave-twcr"
I20240502 07:58:51.036012 9826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "6cc5b9962ed24441bcba0c07b9f5438e"
format_stamp: "Formatted at 2024-05-02 07:58:51 on dist-test-slave-twcr"
I20240502 07:58:51.038759 9826 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240502 07:58:51.040920 9840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:51.041782 9826 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20240502 07:58:51.041880 9826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "6cc5b9962ed24441bcba0c07b9f5438e"
format_stamp: "Formatted at 2024-05-02 07:58:51 on dist-test-slave-twcr"
I20240502 07:58:51.041949 9826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:51.064257 9826 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:51.064720 9826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:51.064844 9826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:51.065256 9826 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:51.066004 9826 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:51.066048 9826 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:51.066093 9826 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:51.066120 9826 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:51.073817 9826 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.68:37379
I20240502 07:58:51.073853 9953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.68:37379 every 8 connection(s)
I20240502 07:58:51.074615 9826 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240502 07:58:51.081025 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 9826
I20240502 07:58:51.081135 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240502 07:58:51.082403 9954 heartbeater.cc:347] Connected to a master server at 127.3.139.126:46479
I20240502 07:58:51.082585 9954 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:51.083027 9954 heartbeater.cc:511] Master 127.3.139.126:46479 requested a full tablet report, sending...
I20240502 07:58:51.083221 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.69:0
--local_ip_for_outbound_sockets=127.3.139.69
--webserver_interface=127.3.139.69
--webserver_port=0
--tserver_master_addrs=127.3.139.126:46479
--builtin_ntp_servers=127.3.139.84:39259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240502 07:58:51.083954 9372 ts_manager.cc:194] Registered new tserver with Master: 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379)
I20240502 07:58:51.084476 9372 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.68:59123
W20240502 07:58:51.183187 9957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:51.183347 9957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:51.183395 9957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:51.186223 9957 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:58:51.186275 9957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:51.186362 9957 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.69
I20240502 07:58:51.189682 9957 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:39259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.69:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.3.139.69
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.3.139.126:46479
--never_fsync=true
--heap_profile_path=/tmp/kudu.9957
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.69
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:51.190332 9957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:51.191258 9957 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:51.198154 9962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:51.198535 9965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:51.198908 9963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:51.199321 9957 server_base.cc:1034] running on GCE node
I20240502 07:58:51.199617 9957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:51.200060 9957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:51.201206 9957 hybrid_clock.cc:648] HybridClock initialized: now 1714636731201193 us; error 24 us; skew 500 ppm
I20240502 07:58:51.202716 9957 webserver.cc:460] Webserver started at http://127.3.139.69:37901/ using document root <none> and password file <none>
I20240502 07:58:51.203204 9957 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:51.203258 9957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:51.203404 9957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:51.204808 9957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "51e82953ed2247fa9ff24638eab3508a"
format_stamp: "Formatted at 2024-05-02 07:58:51 on dist-test-slave-twcr"
I20240502 07:58:51.205212 9957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "51e82953ed2247fa9ff24638eab3508a"
format_stamp: "Formatted at 2024-05-02 07:58:51 on dist-test-slave-twcr"
I20240502 07:58:51.207980 9957 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 07:58:51.209776 9971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:51.210600 9957 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:51.210691 9957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "51e82953ed2247fa9ff24638eab3508a"
format_stamp: "Formatted at 2024-05-02 07:58:51 on dist-test-slave-twcr"
I20240502 07:58:51.210759 9957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:51.233676 9957 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:51.234102 9957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:51.234210 9957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:51.234586 9957 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:51.235318 9957 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:51.235361 9957 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:51.235394 9957 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:51.235414 9957 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:51.242713 9957 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.69:45345
I20240502 07:58:51.242760 10084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.69:45345 every 8 connection(s)
I20240502 07:58:51.243523 9957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240502 07:58:51.247795 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 9957
I20240502 07:58:51.247897 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20240502 07:58:51.250212 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.70:0
--local_ip_for_outbound_sockets=127.3.139.70
--webserver_interface=127.3.139.70
--webserver_port=0
--tserver_master_addrs=127.3.139.126:46479
--builtin_ntp_servers=127.3.139.84:39259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240502 07:58:51.251161 10085 heartbeater.cc:347] Connected to a master server at 127.3.139.126:46479
I20240502 07:58:51.251354 10085 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:51.251833 10085 heartbeater.cc:511] Master 127.3.139.126:46479 requested a full tablet report, sending...
I20240502 07:58:51.252678 9372 ts_manager.cc:194] Registered new tserver with Master: 51e82953ed2247fa9ff24638eab3508a (127.3.139.69:45345)
I20240502 07:58:51.253223 9372 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.69:43675
W20240502 07:58:51.347332 10088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:51.347502 10088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:51.347548 10088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:51.350461 10088 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:58:51.350517 10088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:51.350595 10088 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.70
I20240502 07:58:51.353960 10088 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:39259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.70:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.3.139.70
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.3.139.126:46479
--never_fsync=true
--heap_profile_path=/tmp/kudu.10088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.70
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:51.354651 10088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:51.355636 10088 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:51.363212 10096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:51.363533 10093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:51.363904 10094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:51.364358 10088 server_base.cc:1034] running on GCE node
I20240502 07:58:51.364693 10088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:51.365180 10088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:51.366328 10088 hybrid_clock.cc:648] HybridClock initialized: now 1714636731366316 us; error 26 us; skew 500 ppm
I20240502 07:58:51.367993 10088 webserver.cc:460] Webserver started at http://127.3.139.70:37095/ using document root <none> and password file <none>
I20240502 07:58:51.368482 10088 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:51.368537 10088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:51.368690 10088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:51.370206 10088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/data/instance:
uuid: "8eb9945f989d4be6b51e1531a1365350"
format_stamp: "Formatted at 2024-05-02 07:58:51 on dist-test-slave-twcr"
I20240502 07:58:51.370636 10088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/wal/instance:
uuid: "8eb9945f989d4be6b51e1531a1365350"
format_stamp: "Formatted at 2024-05-02 07:58:51 on dist-test-slave-twcr"
I20240502 07:58:51.373692 10088 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20240502 07:58:51.375661 10102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:51.376538 10088 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:51.376631 10088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "8eb9945f989d4be6b51e1531a1365350"
format_stamp: "Formatted at 2024-05-02 07:58:51 on dist-test-slave-twcr"
I20240502 07:58:51.376703 10088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:51.412921 10088 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:51.413430 10088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:51.413579 10088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:51.414100 10088 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:51.414968 10088 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:51.415012 10088 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:51.415078 10088 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:51.415108 10088 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:51.422451 10088 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.70:45329
I20240502 07:58:51.422474 10215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.70:45329 every 8 connection(s)
I20240502 07:58:51.423264 10088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240502 07:58:51.425019 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 10088
I20240502 07:58:51.425113 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/wal/instance
I20240502 07:58:51.427251 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.71:0
--local_ip_for_outbound_sockets=127.3.139.71
--webserver_interface=127.3.139.71
--webserver_port=0
--tserver_master_addrs=127.3.139.126:46479
--builtin_ntp_servers=127.3.139.84:39259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240502 07:58:51.431562 10216 heartbeater.cc:347] Connected to a master server at 127.3.139.126:46479
I20240502 07:58:51.431779 10216 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:51.432330 10216 heartbeater.cc:511] Master 127.3.139.126:46479 requested a full tablet report, sending...
I20240502 07:58:51.433279 9372 ts_manager.cc:194] Registered new tserver with Master: 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70:45329)
I20240502 07:58:51.433774 9372 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.70:58491
W20240502 07:58:51.526310 10219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:51.526487 10219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:51.526538 10219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:51.529439 10219 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:58:51.529495 10219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:51.529572 10219 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.71
I20240502 07:58:51.532923 10219 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:39259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.71:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.3.139.71
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.3.139.126:46479
--never_fsync=true
--heap_profile_path=/tmp/kudu.10219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.71
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:51.533596 10219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:51.534521 10219 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:51.541669 10224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:51.541985 10227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:51.542361 10225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:51.542750 10219 server_base.cc:1034] running on GCE node
I20240502 07:58:51.543098 10219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:51.543555 10219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:51.544701 10219 hybrid_clock.cc:648] HybridClock initialized: now 1714636731544693 us; error 26 us; skew 500 ppm
I20240502 07:58:51.546347 10219 webserver.cc:460] Webserver started at http://127.3.139.71:36755/ using document root <none> and password file <none>
I20240502 07:58:51.546828 10219 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:51.546901 10219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:51.547106 10219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:51.548592 10219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/data/instance:
uuid: "98bef2e94ade4763a66c1ad68259744a"
format_stamp: "Formatted at 2024-05-02 07:58:51 on dist-test-slave-twcr"
I20240502 07:58:51.548997 10219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/wal/instance:
uuid: "98bef2e94ade4763a66c1ad68259744a"
format_stamp: "Formatted at 2024-05-02 07:58:51 on dist-test-slave-twcr"
I20240502 07:58:51.551820 10219 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20240502 07:58:51.553648 10233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:51.554493 10219 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:51.554584 10219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "98bef2e94ade4763a66c1ad68259744a"
format_stamp: "Formatted at 2024-05-02 07:58:51 on dist-test-slave-twcr"
I20240502 07:58:51.554652 10219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:51.574509 10219 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:51.574951 10219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:51.575107 10219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:51.575582 10219 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:51.576475 10219 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:51.576529 10219 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:51.576575 10219 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:51.576602 10219 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:51.584539 10219 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.71:46577
I20240502 07:58:51.584582 10346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.71:46577 every 8 connection(s)
I20240502 07:58:51.585283 10219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240502 07:58:51.592267 10347 heartbeater.cc:347] Connected to a master server at 127.3.139.126:46479
I20240502 07:58:51.592464 10347 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:51.592772 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 10219
I20240502 07:58:51.592864 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/wal/instance
I20240502 07:58:51.592944 10347 heartbeater.cc:511] Master 127.3.139.126:46479 requested a full tablet report, sending...
I20240502 07:58:51.593257 9561 heartbeater.cc:503] Master 127.3.139.126:46479 was elected leader, sending a full tablet report...
I20240502 07:58:51.593967 9373 ts_manager.cc:194] Registered new tserver with Master: 98bef2e94ade4763a66c1ad68259744a (127.3.139.71:46577)
I20240502 07:58:51.594549 9373 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.71:43079
I20240502 07:58:51.594894 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.72:0
--local_ip_for_outbound_sockets=127.3.139.72
--webserver_interface=127.3.139.72
--webserver_port=0
--tserver_master_addrs=127.3.139.126:46479
--builtin_ntp_servers=127.3.139.84:39259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
W20240502 07:58:51.690768 10350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:51.690933 10350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:51.690999 10350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:51.693876 10350 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:58:51.693933 10350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:51.694020 10350 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.72
I20240502 07:58:51.697321 10350 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:39259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.72:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.3.139.72
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.3.139.126:46479
--never_fsync=true
--heap_profile_path=/tmp/kudu.10350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.72
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:51.697988 10350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:51.698879 10350 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:51.705811 10355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:51.706138 10358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:51.706562 10356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:51.706916 10350 server_base.cc:1034] running on GCE node
I20240502 07:58:51.707263 10350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:51.707710 10350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:51.708847 10350 hybrid_clock.cc:648] HybridClock initialized: now 1714636731708827 us; error 28 us; skew 500 ppm
I20240502 07:58:51.710376 10350 webserver.cc:460] Webserver started at http://127.3.139.72:46209/ using document root <none> and password file <none>
I20240502 07:58:51.710830 10350 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:51.710882 10350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:51.711028 10350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:51.712416 10350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/data/instance:
uuid: "a95f1d3af33f4c879fd290ca917d6329"
format_stamp: "Formatted at 2024-05-02 07:58:51 on dist-test-slave-twcr"
I20240502 07:58:51.712821 10350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/wal/instance:
uuid: "a95f1d3af33f4c879fd290ca917d6329"
format_stamp: "Formatted at 2024-05-02 07:58:51 on dist-test-slave-twcr"
I20240502 07:58:51.715780 10350 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20240502 07:58:51.717624 10364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:51.718456 10350 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:51.718541 10350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "a95f1d3af33f4c879fd290ca917d6329"
format_stamp: "Formatted at 2024-05-02 07:58:51 on dist-test-slave-twcr"
I20240502 07:58:51.718613 10350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:51.738973 10350 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:51.739459 10350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:51.739585 10350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:51.740005 10350 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:51.740760 10350 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:51.740805 10350 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:51.740844 10350 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:51.740871 10350 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:51.747749 9692 heartbeater.cc:503] Master 127.3.139.126:46479 was elected leader, sending a full tablet report...
I20240502 07:58:51.748633 10350 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.72:36271
I20240502 07:58:51.748667 10477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.72:36271 every 8 connection(s)
I20240502 07:58:51.749481 10350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240502 07:58:51.756466 10478 heartbeater.cc:347] Connected to a master server at 127.3.139.126:46479
I20240502 07:58:51.756635 10478 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:51.757071 10478 heartbeater.cc:511] Master 127.3.139.126:46479 requested a full tablet report, sending...
I20240502 07:58:51.757834 9373 ts_manager.cc:194] Registered new tserver with Master: a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72:36271)
I20240502 07:58:51.758355 9373 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.72:45983
I20240502 07:58:51.759476 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 10350
I20240502 07:58:51.759574 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/wal/instance
I20240502 07:58:51.761525 3629 external_mini_cluster.cc:930] 8 TS(s) registered with all masters
I20240502 07:58:51.769621 3629 external_mini_cluster.cc:930] 8 TS(s) registered with all masters
I20240502 07:58:51.769694 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 9957
I20240502 07:58:51.775111 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 10088
I20240502 07:58:51.779695 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 10219
I20240502 07:58:51.783818 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 10350
I20240502 07:58:51.918226 9823 heartbeater.cc:503] Master 127.3.139.126:46479 was elected leader, sending a full tablet report...
I20240502 07:58:52.086107 9954 heartbeater.cc:503] Master 127.3.139.126:46479 was elected leader, sending a full tablet report...
I20240502 07:58:55.539582 9373 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47034:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:58:55.559998 9493 tablet_service.cc:1467] Processing CreateTablet for tablet 92f37161960e4ee894c75d9034434001 (DEFAULT_TABLE table=rebalance_test_table_0 [id=efdfc231272e43a5afefacf0ec75ec57]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:55.560065 9888 tablet_service.cc:1467] Processing CreateTablet for tablet fcdf47a04bdd4b9fa600a0bba59a4542 (DEFAULT_TABLE table=rebalance_test_table_0 [id=efdfc231272e43a5afefacf0ec75ec57]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:55.559998 9626 tablet_service.cc:1467] Processing CreateTablet for tablet fcdf47a04bdd4b9fa600a0bba59a4542 (DEFAULT_TABLE table=rebalance_test_table_0 [id=efdfc231272e43a5afefacf0ec75ec57]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:55.561000 9493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92f37161960e4ee894c75d9034434001. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:55.561003 9626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcdf47a04bdd4b9fa600a0bba59a4542. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:55.561002 9888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcdf47a04bdd4b9fa600a0bba59a4542. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:55.563295 9625 tablet_service.cc:1467] Processing CreateTablet for tablet 686b5cf6614d45c18cc776bd79567667 (DEFAULT_TABLE table=rebalance_test_table_0 [id=efdfc231272e43a5afefacf0ec75ec57]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:55.563294 9887 tablet_service.cc:1467] Processing CreateTablet for tablet 92f37161960e4ee894c75d9034434001 (DEFAULT_TABLE table=rebalance_test_table_0 [id=efdfc231272e43a5afefacf0ec75ec57]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:55.563297 9495 tablet_service.cc:1467] Processing CreateTablet for tablet fcdf47a04bdd4b9fa600a0bba59a4542 (DEFAULT_TABLE table=rebalance_test_table_0 [id=efdfc231272e43a5afefacf0ec75ec57]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:55.563530 9625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 686b5cf6614d45c18cc776bd79567667. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:55.563530 9887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92f37161960e4ee894c75d9034434001. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:55.563547 9495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcdf47a04bdd4b9fa600a0bba59a4542. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:55.566740 10498 tablet_bootstrap.cc:492] T fcdf47a04bdd4b9fa600a0bba59a4542 P a201db721bda4dc3a4422dc8fa9bf72e: Bootstrap starting.
I20240502 07:58:55.559998 9494 tablet_service.cc:1467] Processing CreateTablet for tablet 686b5cf6614d45c18cc776bd79567667 (DEFAULT_TABLE table=rebalance_test_table_0 [id=efdfc231272e43a5afefacf0ec75ec57]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:55.567433 9494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 686b5cf6614d45c18cc776bd79567667. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:55.568923 10498 tablet_bootstrap.cc:654] T fcdf47a04bdd4b9fa600a0bba59a4542 P a201db721bda4dc3a4422dc8fa9bf72e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:55.569689 10498 log.cc:824] T fcdf47a04bdd4b9fa600a0bba59a4542 P a201db721bda4dc3a4422dc8fa9bf72e: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:55.570631 9757 tablet_service.cc:1467] Processing CreateTablet for tablet 686b5cf6614d45c18cc776bd79567667 (DEFAULT_TABLE table=rebalance_test_table_0 [id=efdfc231272e43a5afefacf0ec75ec57]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:55.571359 10500 tablet_bootstrap.cc:492] T 686b5cf6614d45c18cc776bd79567667 P 8337af1943564c0dad18b45a597b805a: Bootstrap starting.
I20240502 07:58:55.571362 9756 tablet_service.cc:1467] Processing CreateTablet for tablet 92f37161960e4ee894c75d9034434001 (DEFAULT_TABLE table=rebalance_test_table_0 [id=efdfc231272e43a5afefacf0ec75ec57]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:55.571444 10498 tablet_bootstrap.cc:492] T fcdf47a04bdd4b9fa600a0bba59a4542 P a201db721bda4dc3a4422dc8fa9bf72e: No bootstrap required, opened a new log
I20240502 07:58:55.571455 9757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 686b5cf6614d45c18cc776bd79567667. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:55.571606 10498 ts_tablet_manager.cc:1367] T fcdf47a04bdd4b9fa600a0bba59a4542 P a201db721bda4dc3a4422dc8fa9bf72e: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20240502 07:58:55.571666 10499 tablet_bootstrap.cc:492] T 92f37161960e4ee894c75d9034434001 P 6cc5b9962ed24441bcba0c07b9f5438e: Bootstrap starting.
I20240502 07:58:55.572486 9756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92f37161960e4ee894c75d9034434001. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:55.574456 10500 tablet_bootstrap.cc:654] T 686b5cf6614d45c18cc776bd79567667 P 8337af1943564c0dad18b45a597b805a: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:55.575208 10500 log.cc:824] T 686b5cf6614d45c18cc776bd79567667 P 8337af1943564c0dad18b45a597b805a: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:55.575397 10498 raft_consensus.cc:348] T fcdf47a04bdd4b9fa600a0bba59a4542 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:55.575713 10498 raft_consensus.cc:374] T fcdf47a04bdd4b9fa600a0bba59a4542 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:55.575809 10498 raft_consensus.cc:729] T fcdf47a04bdd4b9fa600a0bba59a4542 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a201db721bda4dc3a4422dc8fa9bf72e, State: Initialized, Role: FOLLOWER
I20240502 07:58:55.576236 10498 consensus_queue.cc:260] T fcdf47a04bdd4b9fa600a0bba59a4542 P a201db721bda4dc3a4422dc8fa9bf72e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:55.576614 10499 tablet_bootstrap.cc:654] T 92f37161960e4ee894c75d9034434001 P 6cc5b9962ed24441bcba0c07b9f5438e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:55.576995 10503 tablet_bootstrap.cc:492] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3: Bootstrap starting.
I20240502 07:58:55.577255 10498 ts_tablet_manager.cc:1398] T fcdf47a04bdd4b9fa600a0bba59a4542 P a201db721bda4dc3a4422dc8fa9bf72e: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20240502 07:58:55.577306 10499 log.cc:824] T 92f37161960e4ee894c75d9034434001 P 6cc5b9962ed24441bcba0c07b9f5438e: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:55.578791 10503 tablet_bootstrap.cc:654] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:55.579447 10503 log.cc:824] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:55.580281 10498 tablet_bootstrap.cc:492] T 686b5cf6614d45c18cc776bd79567667 P a201db721bda4dc3a4422dc8fa9bf72e: Bootstrap starting.
I20240502 07:58:55.580850 10503 tablet_bootstrap.cc:492] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3: No bootstrap required, opened a new log
I20240502 07:58:55.581007 10503 ts_tablet_manager.cc:1367] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240502 07:58:55.581130 10498 tablet_bootstrap.cc:654] T 686b5cf6614d45c18cc776bd79567667 P a201db721bda4dc3a4422dc8fa9bf72e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:55.582127 10498 tablet_bootstrap.cc:492] T 686b5cf6614d45c18cc776bd79567667 P a201db721bda4dc3a4422dc8fa9bf72e: No bootstrap required, opened a new log
I20240502 07:58:55.582211 10498 ts_tablet_manager.cc:1367] T 686b5cf6614d45c18cc776bd79567667 P a201db721bda4dc3a4422dc8fa9bf72e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:55.582677 10498 raft_consensus.cc:348] T 686b5cf6614d45c18cc776bd79567667 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:55.582767 10498 raft_consensus.cc:374] T 686b5cf6614d45c18cc776bd79567667 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:55.582793 10498 raft_consensus.cc:729] T 686b5cf6614d45c18cc776bd79567667 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a201db721bda4dc3a4422dc8fa9bf72e, State: Initialized, Role: FOLLOWER
I20240502 07:58:55.582909 10498 consensus_queue.cc:260] T 686b5cf6614d45c18cc776bd79567667 P a201db721bda4dc3a4422dc8fa9bf72e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:55.583237 10498 ts_tablet_manager.cc:1398] T 686b5cf6614d45c18cc776bd79567667 P a201db721bda4dc3a4422dc8fa9bf72e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:55.583390 10498 tablet_bootstrap.cc:492] T 92f37161960e4ee894c75d9034434001 P a201db721bda4dc3a4422dc8fa9bf72e: Bootstrap starting.
I20240502 07:58:55.584044 10498 tablet_bootstrap.cc:654] T 92f37161960e4ee894c75d9034434001 P a201db721bda4dc3a4422dc8fa9bf72e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:55.584312 10503 raft_consensus.cc:348] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:58:55.584499 10503 raft_consensus.cc:374] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20240502 07:58:55.584527 9562 tablet.cc:2369] T 686b5cf6614d45c18cc776bd79567667 P a201db721bda4dc3a4422dc8fa9bf72e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 07:58:55.584548 10503 raft_consensus.cc:729] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1472c64688914149a318c7b8850a3df3, State: Initialized, Role: FOLLOWER
I20240502 07:58:55.585006 10503 consensus_queue.cc:260] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:58:55.585568 10498 tablet_bootstrap.cc:492] T 92f37161960e4ee894c75d9034434001 P a201db721bda4dc3a4422dc8fa9bf72e: No bootstrap required, opened a new log
I20240502 07:58:55.585615 10500 tablet_bootstrap.cc:492] T 686b5cf6614d45c18cc776bd79567667 P 8337af1943564c0dad18b45a597b805a: No bootstrap required, opened a new log
I20240502 07:58:55.585641 10498 ts_tablet_manager.cc:1367] T 92f37161960e4ee894c75d9034434001 P a201db721bda4dc3a4422dc8fa9bf72e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 07:58:55.585770 10500 ts_tablet_manager.cc:1367] T 686b5cf6614d45c18cc776bd79567667 P 8337af1943564c0dad18b45a597b805a: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.003s
I20240502 07:58:55.585893 10503 ts_tablet_manager.cc:1398] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20240502 07:58:55.586021 10498 raft_consensus.cc:348] T 92f37161960e4ee894c75d9034434001 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:58:55.586104 10498 raft_consensus.cc:374] T 92f37161960e4ee894c75d9034434001 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:55.586131 10498 raft_consensus.cc:729] T 92f37161960e4ee894c75d9034434001 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a201db721bda4dc3a4422dc8fa9bf72e, State: Initialized, Role: FOLLOWER
I20240502 07:58:55.586202 10503 tablet_bootstrap.cc:492] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3: Bootstrap starting.
I20240502 07:58:55.586246 10499 tablet_bootstrap.cc:492] T 92f37161960e4ee894c75d9034434001 P 6cc5b9962ed24441bcba0c07b9f5438e: No bootstrap required, opened a new log
I20240502 07:58:55.586243 10498 consensus_queue.cc:260] T 92f37161960e4ee894c75d9034434001 P a201db721bda4dc3a4422dc8fa9bf72e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:58:55.586395 10499 ts_tablet_manager.cc:1367] T 92f37161960e4ee894c75d9034434001 P 6cc5b9962ed24441bcba0c07b9f5438e: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.000s
I20240502 07:58:55.586555 10498 ts_tablet_manager.cc:1398] T 92f37161960e4ee894c75d9034434001 P a201db721bda4dc3a4422dc8fa9bf72e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:55.587013 10503 tablet_bootstrap.cc:654] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:55.589893 10500 raft_consensus.cc:348] T 686b5cf6614d45c18cc776bd79567667 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:55.589905 10499 raft_consensus.cc:348] T 92f37161960e4ee894c75d9034434001 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:58:55.590093 10499 raft_consensus.cc:374] T 92f37161960e4ee894c75d9034434001 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:55.590094 10500 raft_consensus.cc:374] T 686b5cf6614d45c18cc776bd79567667 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:55.590154 10500 raft_consensus.cc:729] T 686b5cf6614d45c18cc776bd79567667 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8337af1943564c0dad18b45a597b805a, State: Initialized, Role: FOLLOWER
I20240502 07:58:55.590178 10499 raft_consensus.cc:729] T 92f37161960e4ee894c75d9034434001 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cc5b9962ed24441bcba0c07b9f5438e, State: Initialized, Role: FOLLOWER
I20240502 07:58:55.590551 10500 consensus_queue.cc:260] T 686b5cf6614d45c18cc776bd79567667 P 8337af1943564c0dad18b45a597b805a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:55.590641 10499 consensus_queue.cc:260] T 92f37161960e4ee894c75d9034434001 P 6cc5b9962ed24441bcba0c07b9f5438e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:58:55.591336 10500 ts_tablet_manager.cc:1398] T 686b5cf6614d45c18cc776bd79567667 P 8337af1943564c0dad18b45a597b805a: Time spent starting tablet: real 0.006s user 0.004s sys 0.001s
I20240502 07:58:55.591470 10499 ts_tablet_manager.cc:1398] T 92f37161960e4ee894c75d9034434001 P 6cc5b9962ed24441bcba0c07b9f5438e: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20240502 07:58:55.591697 10500 tablet_bootstrap.cc:492] T fcdf47a04bdd4b9fa600a0bba59a4542 P 8337af1943564c0dad18b45a597b805a: Bootstrap starting.
I20240502 07:58:55.591912 10499 tablet_bootstrap.cc:492] T fcdf47a04bdd4b9fa600a0bba59a4542 P 6cc5b9962ed24441bcba0c07b9f5438e: Bootstrap starting.
I20240502 07:58:55.592609 10500 tablet_bootstrap.cc:654] T fcdf47a04bdd4b9fa600a0bba59a4542 P 8337af1943564c0dad18b45a597b805a: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:55.592873 10499 tablet_bootstrap.cc:654] T fcdf47a04bdd4b9fa600a0bba59a4542 P 6cc5b9962ed24441bcba0c07b9f5438e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:55.593645 10500 tablet_bootstrap.cc:492] T fcdf47a04bdd4b9fa600a0bba59a4542 P 8337af1943564c0dad18b45a597b805a: No bootstrap required, opened a new log
I20240502 07:58:55.593772 10500 ts_tablet_manager.cc:1367] T fcdf47a04bdd4b9fa600a0bba59a4542 P 8337af1943564c0dad18b45a597b805a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:55.593930 10503 tablet_bootstrap.cc:492] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3: No bootstrap required, opened a new log
I20240502 07:58:55.594022 10503 ts_tablet_manager.cc:1367] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 07:58:55.594343 10499 tablet_bootstrap.cc:492] T fcdf47a04bdd4b9fa600a0bba59a4542 P 6cc5b9962ed24441bcba0c07b9f5438e: No bootstrap required, opened a new log
I20240502 07:58:55.594455 10499 ts_tablet_manager.cc:1367] T fcdf47a04bdd4b9fa600a0bba59a4542 P 6cc5b9962ed24441bcba0c07b9f5438e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:55.594448 10500 raft_consensus.cc:348] T fcdf47a04bdd4b9fa600a0bba59a4542 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:55.594527 10503 raft_consensus.cc:348] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:55.594619 10500 raft_consensus.cc:374] T fcdf47a04bdd4b9fa600a0bba59a4542 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:55.594671 10500 raft_consensus.cc:729] T fcdf47a04bdd4b9fa600a0bba59a4542 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8337af1943564c0dad18b45a597b805a, State: Initialized, Role: FOLLOWER
I20240502 07:58:55.594671 10503 raft_consensus.cc:374] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:55.594707 10503 raft_consensus.cc:729] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1472c64688914149a318c7b8850a3df3, State: Initialized, Role: FOLLOWER
I20240502 07:58:55.594811 10500 consensus_queue.cc:260] T fcdf47a04bdd4b9fa600a0bba59a4542 P 8337af1943564c0dad18b45a597b805a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:55.594861 10503 consensus_queue.cc:260] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:55.594941 10499 raft_consensus.cc:348] T fcdf47a04bdd4b9fa600a0bba59a4542 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:55.595081 10499 raft_consensus.cc:374] T fcdf47a04bdd4b9fa600a0bba59a4542 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:55.595145 10503 ts_tablet_manager.cc:1398] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:55.595149 10499 raft_consensus.cc:729] T fcdf47a04bdd4b9fa600a0bba59a4542 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cc5b9962ed24441bcba0c07b9f5438e, State: Initialized, Role: FOLLOWER
I20240502 07:58:55.595183 10500 ts_tablet_manager.cc:1398] T fcdf47a04bdd4b9fa600a0bba59a4542 P 8337af1943564c0dad18b45a597b805a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:55.595265 10499 consensus_queue.cc:260] T fcdf47a04bdd4b9fa600a0bba59a4542 P 6cc5b9962ed24441bcba0c07b9f5438e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:55.595796 10499 ts_tablet_manager.cc:1398] T fcdf47a04bdd4b9fa600a0bba59a4542 P 6cc5b9962ed24441bcba0c07b9f5438e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20240502 07:58:55.660562 9824 tablet.cc:2369] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 07:58:55.661159 10510 raft_consensus.cc:482] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:55.661329 10510 raft_consensus.cc:504] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:58:55.662381 10510 leader_election.cc:288] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379), a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551)
I20240502 07:58:55.665967 9908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92f37161960e4ee894c75d9034434001" candidate_uuid: "1472c64688914149a318c7b8850a3df3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" is_pre_election: true
I20240502 07:58:55.665980 9515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92f37161960e4ee894c75d9034434001" candidate_uuid: "1472c64688914149a318c7b8850a3df3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" is_pre_election: true
I20240502 07:58:55.666227 9515 raft_consensus.cc:2454] T 92f37161960e4ee894c75d9034434001 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1472c64688914149a318c7b8850a3df3 in term 0.
I20240502 07:58:55.666229 9908 raft_consensus.cc:2454] T 92f37161960e4ee894c75d9034434001 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1472c64688914149a318c7b8850a3df3 in term 0.
I20240502 07:58:55.666625 9711 leader_election.cc:302] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1472c64688914149a318c7b8850a3df3, 6cc5b9962ed24441bcba0c07b9f5438e; no voters:
I20240502 07:58:55.666862 10510 raft_consensus.cc:2781] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:55.666925 10510 raft_consensus.cc:482] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:55.666954 10510 raft_consensus.cc:3037] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:55.667855 10510 raft_consensus.cc:504] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:58:55.668198 10510 leader_election.cc:288] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [CANDIDATE]: Term 1 election: Requested vote from peers 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379), a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551)
I20240502 07:58:55.668473 9908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92f37161960e4ee894c75d9034434001" candidate_uuid: "1472c64688914149a318c7b8850a3df3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cc5b9962ed24441bcba0c07b9f5438e"
I20240502 07:58:55.668517 9515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92f37161960e4ee894c75d9034434001" candidate_uuid: "1472c64688914149a318c7b8850a3df3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a201db721bda4dc3a4422dc8fa9bf72e"
I20240502 07:58:55.668598 9908 raft_consensus.cc:3037] T 92f37161960e4ee894c75d9034434001 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:55.668609 9515 raft_consensus.cc:3037] T 92f37161960e4ee894c75d9034434001 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:55.669673 9908 raft_consensus.cc:2454] T 92f37161960e4ee894c75d9034434001 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1472c64688914149a318c7b8850a3df3 in term 1.
I20240502 07:58:55.669943 9711 leader_election.cc:302] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1472c64688914149a318c7b8850a3df3, 6cc5b9962ed24441bcba0c07b9f5438e; no voters:
I20240502 07:58:55.670027 9515 raft_consensus.cc:2454] T 92f37161960e4ee894c75d9034434001 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1472c64688914149a318c7b8850a3df3 in term 1.
I20240502 07:58:55.670105 10510 raft_consensus.cc:2781] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:55.670336 10510 raft_consensus.cc:686] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Becoming Leader. State: Replica: 1472c64688914149a318c7b8850a3df3, State: Running, Role: LEADER
I20240502 07:58:55.670694 10510 consensus_queue.cc:237] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:58:55.673700 9373 catalog_manager.cc:5484] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1472c64688914149a318c7b8850a3df3 (127.3.139.67). New cstate: current_term: 1 leader_uuid: "1472c64688914149a318c7b8850a3df3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:55.736564 10517 raft_consensus.cc:482] T fcdf47a04bdd4b9fa600a0bba59a4542 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:55.736764 10517 raft_consensus.cc:504] T fcdf47a04bdd4b9fa600a0bba59a4542 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:55.737952 10517 leader_election.cc:288] T fcdf47a04bdd4b9fa600a0bba59a4542 P 8337af1943564c0dad18b45a597b805a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379), a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551)
W20240502 07:58:55.739365 9693 tablet.cc:2369] T 686b5cf6614d45c18cc776bd79567667 P 8337af1943564c0dad18b45a597b805a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 07:58:55.741933 9515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcdf47a04bdd4b9fa600a0bba59a4542" candidate_uuid: "8337af1943564c0dad18b45a597b805a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" is_pre_election: true
I20240502 07:58:55.742118 9515 raft_consensus.cc:2454] T fcdf47a04bdd4b9fa600a0bba59a4542 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8337af1943564c0dad18b45a597b805a in term 0.
I20240502 07:58:55.742453 9582 leader_election.cc:302] T fcdf47a04bdd4b9fa600a0bba59a4542 P 8337af1943564c0dad18b45a597b805a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8337af1943564c0dad18b45a597b805a, a201db721bda4dc3a4422dc8fa9bf72e; no voters:
I20240502 07:58:55.742679 10517 raft_consensus.cc:2781] T fcdf47a04bdd4b9fa600a0bba59a4542 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:55.742668 9908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcdf47a04bdd4b9fa600a0bba59a4542" candidate_uuid: "8337af1943564c0dad18b45a597b805a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" is_pre_election: true
I20240502 07:58:55.742749 10517 raft_consensus.cc:482] T fcdf47a04bdd4b9fa600a0bba59a4542 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:55.742784 10517 raft_consensus.cc:3037] T fcdf47a04bdd4b9fa600a0bba59a4542 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:55.742790 9908 raft_consensus.cc:2454] T fcdf47a04bdd4b9fa600a0bba59a4542 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8337af1943564c0dad18b45a597b805a in term 0.
I20240502 07:58:55.745395 10517 raft_consensus.cc:504] T fcdf47a04bdd4b9fa600a0bba59a4542 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:55.745724 10517 leader_election.cc:288] T fcdf47a04bdd4b9fa600a0bba59a4542 P 8337af1943564c0dad18b45a597b805a [CANDIDATE]: Term 1 election: Requested vote from peers 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379), a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551)
I20240502 07:58:55.746062 9908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcdf47a04bdd4b9fa600a0bba59a4542" candidate_uuid: "8337af1943564c0dad18b45a597b805a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cc5b9962ed24441bcba0c07b9f5438e"
I20240502 07:58:55.746053 9515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcdf47a04bdd4b9fa600a0bba59a4542" candidate_uuid: "8337af1943564c0dad18b45a597b805a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a201db721bda4dc3a4422dc8fa9bf72e"
I20240502 07:58:55.746153 9908 raft_consensus.cc:3037] T fcdf47a04bdd4b9fa600a0bba59a4542 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:55.746165 9515 raft_consensus.cc:3037] T fcdf47a04bdd4b9fa600a0bba59a4542 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:55.746832 9908 raft_consensus.cc:2454] T fcdf47a04bdd4b9fa600a0bba59a4542 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8337af1943564c0dad18b45a597b805a in term 1.
I20240502 07:58:55.746876 9515 raft_consensus.cc:2454] T fcdf47a04bdd4b9fa600a0bba59a4542 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8337af1943564c0dad18b45a597b805a in term 1.
I20240502 07:58:55.747124 9580 leader_election.cc:302] T fcdf47a04bdd4b9fa600a0bba59a4542 P 8337af1943564c0dad18b45a597b805a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cc5b9962ed24441bcba0c07b9f5438e, 8337af1943564c0dad18b45a597b805a; no voters:
I20240502 07:58:55.747303 10517 raft_consensus.cc:2781] T fcdf47a04bdd4b9fa600a0bba59a4542 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:55.747404 10517 raft_consensus.cc:686] T fcdf47a04bdd4b9fa600a0bba59a4542 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Becoming Leader. State: Replica: 8337af1943564c0dad18b45a597b805a, State: Running, Role: LEADER
I20240502 07:58:55.747696 10517 consensus_queue.cc:237] T fcdf47a04bdd4b9fa600a0bba59a4542 P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:55.750533 9373 catalog_manager.cc:5484] T fcdf47a04bdd4b9fa600a0bba59a4542 P 8337af1943564c0dad18b45a597b805a reported cstate change: term changed from 0 to 1, leader changed from <none> to 8337af1943564c0dad18b45a597b805a (127.3.139.66). New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: HEALTHY } } }
I20240502 07:58:55.808637 10510 raft_consensus.cc:482] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:55.808777 10510 raft_consensus.cc:504] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:55.809314 10510 leader_election.cc:288] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551), 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667)
I20240502 07:58:55.809760 9515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "686b5cf6614d45c18cc776bd79567667" candidate_uuid: "1472c64688914149a318c7b8850a3df3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" is_pre_election: true
I20240502 07:58:55.809932 9515 raft_consensus.cc:2454] T 686b5cf6614d45c18cc776bd79567667 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1472c64688914149a318c7b8850a3df3 in term 0.
I20240502 07:58:55.810212 9713 leader_election.cc:302] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1472c64688914149a318c7b8850a3df3, a201db721bda4dc3a4422dc8fa9bf72e; no voters:
I20240502 07:58:55.810397 10510 raft_consensus.cc:2781] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:55.810457 10510 raft_consensus.cc:482] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:55.810482 10510 raft_consensus.cc:3037] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:55.811298 10510 raft_consensus.cc:504] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:55.811640 10510 leader_election.cc:288] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 [CANDIDATE]: Term 1 election: Requested vote from peers a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551), 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667)
I20240502 07:58:55.811976 9515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "686b5cf6614d45c18cc776bd79567667" candidate_uuid: "1472c64688914149a318c7b8850a3df3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a201db721bda4dc3a4422dc8fa9bf72e"
I20240502 07:58:55.812085 9515 raft_consensus.cc:3037] T 686b5cf6614d45c18cc776bd79567667 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:55.812873 9515 raft_consensus.cc:2454] T 686b5cf6614d45c18cc776bd79567667 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1472c64688914149a318c7b8850a3df3 in term 1.
I20240502 07:58:55.812892 9645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "686b5cf6614d45c18cc776bd79567667" candidate_uuid: "1472c64688914149a318c7b8850a3df3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8337af1943564c0dad18b45a597b805a"
I20240502 07:58:55.813103 9645 raft_consensus.cc:3037] T 686b5cf6614d45c18cc776bd79567667 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:55.813108 9646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "686b5cf6614d45c18cc776bd79567667" candidate_uuid: "1472c64688914149a318c7b8850a3df3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8337af1943564c0dad18b45a597b805a" is_pre_election: true
I20240502 07:58:55.813268 9713 leader_election.cc:302] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1472c64688914149a318c7b8850a3df3, a201db721bda4dc3a4422dc8fa9bf72e; no voters:
I20240502 07:58:55.813427 10510 raft_consensus.cc:2781] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:55.813510 10510 raft_consensus.cc:686] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Becoming Leader. State: Replica: 1472c64688914149a318c7b8850a3df3, State: Running, Role: LEADER
I20240502 07:58:55.813653 10510 consensus_queue.cc:237] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:55.814133 9645 raft_consensus.cc:2454] T 686b5cf6614d45c18cc776bd79567667 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1472c64688914149a318c7b8850a3df3 in term 1.
I20240502 07:58:55.815358 9373 catalog_manager.cc:5484] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1472c64688914149a318c7b8850a3df3 (127.3.139.67). New cstate: current_term: 1 leader_uuid: "1472c64688914149a318c7b8850a3df3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: UNKNOWN } } }
W20240502 07:58:55.833411 9955 tablet.cc:2369] T 92f37161960e4ee894c75d9034434001 P 6cc5b9962ed24441bcba0c07b9f5438e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240502 07:58:55.951498 10526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:55.951719 10526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:55.954599 10526 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_0'
I20240502 07:58:56.033856 9908 raft_consensus.cc:1261] T fcdf47a04bdd4b9fa600a0bba59a4542 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:56.034209 9645 raft_consensus.cc:1261] T 686b5cf6614d45c18cc776bd79567667 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:56.034673 9515 raft_consensus.cc:1261] T fcdf47a04bdd4b9fa600a0bba59a4542 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:56.034682 9514 raft_consensus.cc:1261] T 686b5cf6614d45c18cc776bd79567667 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:56.035287 10517 consensus_queue.cc:1035] T fcdf47a04bdd4b9fa600a0bba59a4542 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:56.034991 10510 consensus_queue.cc:1035] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:56.036120 9514 raft_consensus.cc:1261] T 92f37161960e4ee894c75d9034434001 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:56.036296 10520 consensus_queue.cc:1035] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:56.036576 10520 consensus_queue.cc:1035] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:56.037253 9908 raft_consensus.cc:1261] T 92f37161960e4ee894c75d9034434001 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:56.037376 10513 consensus_queue.cc:1035] T fcdf47a04bdd4b9fa600a0bba59a4542 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:56.041044 10510 consensus_queue.cc:1035] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:56.045768 10557 mvcc.cc:205] Tried to move back new op lower bound from 7023152070787682304 to 7023152069624995840. Current Snapshot: MvccSnapshot[applied={T|T < 7023152070787682304}]
I20240502 07:58:56.049537 10564 mvcc.cc:205] Tried to move back new op lower bound from 7023152070789509120 to 7023152069310029824. Current Snapshot: MvccSnapshot[applied={T|T < 7023152070789509120}]
I20240502 07:58:56.051620 10555 mvcc.cc:205] Tried to move back new op lower bound from 7023152070789509120 to 7023152069310029824. Current Snapshot: MvccSnapshot[applied={T|T < 7023152070789509120}]
I20240502 07:58:56.067631 10561 mvcc.cc:205] Tried to move back new op lower bound from 7023152070789509120 to 7023152069310029824. Current Snapshot: MvccSnapshot[applied={T|T < 7023152070789509120}]
INSERT report
rows total: 2000
time total: 109.237 ms
time per row: 0.0546184 ms
I20240502 07:58:56.106501 9371 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47034:
name: "rebalance_test_table_1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:58:56.111588 9757 tablet_service.cc:1467] Processing CreateTablet for tablet d562a250cee247a48e573b25ecf00429 (DEFAULT_TABLE table=rebalance_test_table_1 [id=982b41384c8f44b99bbe8eecffb1cd87]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:56.111794 9888 tablet_service.cc:1467] Processing CreateTablet for tablet d562a250cee247a48e573b25ecf00429 (DEFAULT_TABLE table=rebalance_test_table_1 [id=982b41384c8f44b99bbe8eecffb1cd87]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:56.111933 9757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d562a250cee247a48e573b25ecf00429. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:56.112066 9888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d562a250cee247a48e573b25ecf00429. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:56.112432 9493 tablet_service.cc:1467] Processing CreateTablet for tablet e2ba8220f81a4d6cb3611ca53db91fe1 (DEFAULT_TABLE table=rebalance_test_table_1 [id=982b41384c8f44b99bbe8eecffb1cd87]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:56.112665 9493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2ba8220f81a4d6cb3611ca53db91fe1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:56.113692 10588 tablet_bootstrap.cc:492] T d562a250cee247a48e573b25ecf00429 P 1472c64688914149a318c7b8850a3df3: Bootstrap starting.
I20240502 07:58:56.114931 9756 tablet_service.cc:1467] Processing CreateTablet for tablet e2ba8220f81a4d6cb3611ca53db91fe1 (DEFAULT_TABLE table=rebalance_test_table_1 [id=982b41384c8f44b99bbe8eecffb1cd87]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:56.114953 9626 tablet_service.cc:1467] Processing CreateTablet for tablet d562a250cee247a48e573b25ecf00429 (DEFAULT_TABLE table=rebalance_test_table_1 [id=982b41384c8f44b99bbe8eecffb1cd87]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:56.115222 9626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d562a250cee247a48e573b25ecf00429. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:56.115231 9756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2ba8220f81a4d6cb3611ca53db91fe1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:56.115279 10588 tablet_bootstrap.cc:654] T d562a250cee247a48e573b25ecf00429 P 1472c64688914149a318c7b8850a3df3: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:56.116257 9757 tablet_service.cc:1467] Processing CreateTablet for tablet 50f059668c6b4205a15d2c076e2789b0 (DEFAULT_TABLE table=rebalance_test_table_1 [id=982b41384c8f44b99bbe8eecffb1cd87]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:56.116328 10588 tablet_bootstrap.cc:492] T d562a250cee247a48e573b25ecf00429 P 1472c64688914149a318c7b8850a3df3: No bootstrap required, opened a new log
I20240502 07:58:56.116405 10588 ts_tablet_manager.cc:1367] T d562a250cee247a48e573b25ecf00429 P 1472c64688914149a318c7b8850a3df3: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:56.116458 9757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50f059668c6b4205a15d2c076e2789b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:56.116824 10588 raft_consensus.cc:348] T d562a250cee247a48e573b25ecf00429 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:56.116932 10588 raft_consensus.cc:374] T d562a250cee247a48e573b25ecf00429 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:56.116964 10588 raft_consensus.cc:729] T d562a250cee247a48e573b25ecf00429 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1472c64688914149a318c7b8850a3df3, State: Initialized, Role: FOLLOWER
I20240502 07:58:56.117066 10588 consensus_queue.cc:260] T d562a250cee247a48e573b25ecf00429 P 1472c64688914149a318c7b8850a3df3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:56.117331 10589 tablet_bootstrap.cc:492] T d562a250cee247a48e573b25ecf00429 P 6cc5b9962ed24441bcba0c07b9f5438e: Bootstrap starting.
I20240502 07:58:56.117336 10588 ts_tablet_manager.cc:1398] T d562a250cee247a48e573b25ecf00429 P 1472c64688914149a318c7b8850a3df3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:56.117475 10588 tablet_bootstrap.cc:492] T e2ba8220f81a4d6cb3611ca53db91fe1 P 1472c64688914149a318c7b8850a3df3: Bootstrap starting.
I20240502 07:58:56.118213 10589 tablet_bootstrap.cc:654] T d562a250cee247a48e573b25ecf00429 P 6cc5b9962ed24441bcba0c07b9f5438e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:56.118314 10588 tablet_bootstrap.cc:654] T e2ba8220f81a4d6cb3611ca53db91fe1 P 1472c64688914149a318c7b8850a3df3: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:56.118755 9625 tablet_service.cc:1467] Processing CreateTablet for tablet e2ba8220f81a4d6cb3611ca53db91fe1 (DEFAULT_TABLE table=rebalance_test_table_1 [id=982b41384c8f44b99bbe8eecffb1cd87]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:56.118978 9625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2ba8220f81a4d6cb3611ca53db91fe1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:56.119318 10589 tablet_bootstrap.cc:492] T d562a250cee247a48e573b25ecf00429 P 6cc5b9962ed24441bcba0c07b9f5438e: No bootstrap required, opened a new log
I20240502 07:58:56.119406 10589 ts_tablet_manager.cc:1367] T d562a250cee247a48e573b25ecf00429 P 6cc5b9962ed24441bcba0c07b9f5438e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:56.119832 10589 raft_consensus.cc:348] T d562a250cee247a48e573b25ecf00429 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:56.119935 10589 raft_consensus.cc:374] T d562a250cee247a48e573b25ecf00429 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:56.119963 10589 raft_consensus.cc:729] T d562a250cee247a48e573b25ecf00429 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cc5b9962ed24441bcba0c07b9f5438e, State: Initialized, Role: FOLLOWER
I20240502 07:58:56.120030 9626 tablet_service.cc:1467] Processing CreateTablet for tablet 50f059668c6b4205a15d2c076e2789b0 (DEFAULT_TABLE table=rebalance_test_table_1 [id=982b41384c8f44b99bbe8eecffb1cd87]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:56.120080 10589 consensus_queue.cc:260] T d562a250cee247a48e573b25ecf00429 P 6cc5b9962ed24441bcba0c07b9f5438e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:56.120215 9626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50f059668c6b4205a15d2c076e2789b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:56.120355 10589 ts_tablet_manager.cc:1398] T d562a250cee247a48e573b25ecf00429 P 6cc5b9962ed24441bcba0c07b9f5438e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:56.120556 10588 tablet_bootstrap.cc:492] T e2ba8220f81a4d6cb3611ca53db91fe1 P 1472c64688914149a318c7b8850a3df3: No bootstrap required, opened a new log
I20240502 07:58:56.120621 10588 ts_tablet_manager.cc:1367] T e2ba8220f81a4d6cb3611ca53db91fe1 P 1472c64688914149a318c7b8850a3df3: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:58:56.121006 10588 raft_consensus.cc:348] T e2ba8220f81a4d6cb3611ca53db91fe1 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:56.121107 10588 raft_consensus.cc:374] T e2ba8220f81a4d6cb3611ca53db91fe1 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:56.121138 10588 raft_consensus.cc:729] T e2ba8220f81a4d6cb3611ca53db91fe1 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1472c64688914149a318c7b8850a3df3, State: Initialized, Role: FOLLOWER
I20240502 07:58:56.121255 10588 consensus_queue.cc:260] T e2ba8220f81a4d6cb3611ca53db91fe1 P 1472c64688914149a318c7b8850a3df3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:56.121327 10592 tablet_bootstrap.cc:492] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a: Bootstrap starting.
I20240502 07:58:56.121511 10588 ts_tablet_manager.cc:1398] T e2ba8220f81a4d6cb3611ca53db91fe1 P 1472c64688914149a318c7b8850a3df3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:56.121706 10588 tablet_bootstrap.cc:492] T 50f059668c6b4205a15d2c076e2789b0 P 1472c64688914149a318c7b8850a3df3: Bootstrap starting.
I20240502 07:58:56.122114 10592 tablet_bootstrap.cc:654] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:56.122504 10588 tablet_bootstrap.cc:654] T 50f059668c6b4205a15d2c076e2789b0 P 1472c64688914149a318c7b8850a3df3: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:56.123163 10590 tablet_bootstrap.cc:492] T e2ba8220f81a4d6cb3611ca53db91fe1 P a201db721bda4dc3a4422dc8fa9bf72e: Bootstrap starting.
I20240502 07:58:56.123525 9888 tablet_service.cc:1467] Processing CreateTablet for tablet 50f059668c6b4205a15d2c076e2789b0 (DEFAULT_TABLE table=rebalance_test_table_1 [id=982b41384c8f44b99bbe8eecffb1cd87]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:56.123803 9888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50f059668c6b4205a15d2c076e2789b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:56.124105 10590 tablet_bootstrap.cc:654] T e2ba8220f81a4d6cb3611ca53db91fe1 P a201db721bda4dc3a4422dc8fa9bf72e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:56.126318 10592 tablet_bootstrap.cc:492] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a: No bootstrap required, opened a new log
I20240502 07:58:56.126498 10592 ts_tablet_manager.cc:1367] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:58:56.126606 10590 tablet_bootstrap.cc:492] T e2ba8220f81a4d6cb3611ca53db91fe1 P a201db721bda4dc3a4422dc8fa9bf72e: No bootstrap required, opened a new log
I20240502 07:58:56.126643 10588 tablet_bootstrap.cc:492] T 50f059668c6b4205a15d2c076e2789b0 P 1472c64688914149a318c7b8850a3df3: No bootstrap required, opened a new log
I20240502 07:58:56.126688 10590 ts_tablet_manager.cc:1367] T e2ba8220f81a4d6cb3611ca53db91fe1 P a201db721bda4dc3a4422dc8fa9bf72e: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:58:56.126703 10588 ts_tablet_manager.cc:1367] T 50f059668c6b4205a15d2c076e2789b0 P 1472c64688914149a318c7b8850a3df3: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 07:58:56.126935 10592 raft_consensus.cc:348] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:56.127035 10592 raft_consensus.cc:374] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:56.127096 10592 raft_consensus.cc:729] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8337af1943564c0dad18b45a597b805a, State: Initialized, Role: FOLLOWER
I20240502 07:58:56.127118 10588 raft_consensus.cc:348] T 50f059668c6b4205a15d2c076e2789b0 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:56.127118 10590 raft_consensus.cc:348] T e2ba8220f81a4d6cb3611ca53db91fe1 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:56.127210 10590 raft_consensus.cc:374] T e2ba8220f81a4d6cb3611ca53db91fe1 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:56.127210 10588 raft_consensus.cc:374] T 50f059668c6b4205a15d2c076e2789b0 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:56.127242 10588 raft_consensus.cc:729] T 50f059668c6b4205a15d2c076e2789b0 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1472c64688914149a318c7b8850a3df3, State: Initialized, Role: FOLLOWER
I20240502 07:58:56.127242 10590 raft_consensus.cc:729] T e2ba8220f81a4d6cb3611ca53db91fe1 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a201db721bda4dc3a4422dc8fa9bf72e, State: Initialized, Role: FOLLOWER
I20240502 07:58:56.127207 10592 consensus_queue.cc:260] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:56.127357 10590 consensus_queue.cc:260] T e2ba8220f81a4d6cb3611ca53db91fe1 P a201db721bda4dc3a4422dc8fa9bf72e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:56.127368 10588 consensus_queue.cc:260] T 50f059668c6b4205a15d2c076e2789b0 P 1472c64688914149a318c7b8850a3df3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:56.127450 10592 ts_tablet_manager.cc:1398] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:56.127609 10592 tablet_bootstrap.cc:492] T e2ba8220f81a4d6cb3611ca53db91fe1 P 8337af1943564c0dad18b45a597b805a: Bootstrap starting.
I20240502 07:58:56.127606 10590 ts_tablet_manager.cc:1398] T e2ba8220f81a4d6cb3611ca53db91fe1 P a201db721bda4dc3a4422dc8fa9bf72e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:56.127753 10588 ts_tablet_manager.cc:1398] T 50f059668c6b4205a15d2c076e2789b0 P 1472c64688914149a318c7b8850a3df3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:56.128387 10592 tablet_bootstrap.cc:654] T e2ba8220f81a4d6cb3611ca53db91fe1 P 8337af1943564c0dad18b45a597b805a: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:56.131361 10589 tablet_bootstrap.cc:492] T 50f059668c6b4205a15d2c076e2789b0 P 6cc5b9962ed24441bcba0c07b9f5438e: Bootstrap starting.
I20240502 07:58:56.131371 10592 tablet_bootstrap.cc:492] T e2ba8220f81a4d6cb3611ca53db91fe1 P 8337af1943564c0dad18b45a597b805a: No bootstrap required, opened a new log
I20240502 07:58:56.131450 10592 ts_tablet_manager.cc:1367] T e2ba8220f81a4d6cb3611ca53db91fe1 P 8337af1943564c0dad18b45a597b805a: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240502 07:58:56.131836 10592 raft_consensus.cc:348] T e2ba8220f81a4d6cb3611ca53db91fe1 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:56.131939 10592 raft_consensus.cc:374] T e2ba8220f81a4d6cb3611ca53db91fe1 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:56.131970 10592 raft_consensus.cc:729] T e2ba8220f81a4d6cb3611ca53db91fe1 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8337af1943564c0dad18b45a597b805a, State: Initialized, Role: FOLLOWER
I20240502 07:58:56.132102 10592 consensus_queue.cc:260] T e2ba8220f81a4d6cb3611ca53db91fe1 P 8337af1943564c0dad18b45a597b805a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:56.132201 10589 tablet_bootstrap.cc:654] T 50f059668c6b4205a15d2c076e2789b0 P 6cc5b9962ed24441bcba0c07b9f5438e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:56.132408 10592 ts_tablet_manager.cc:1398] T e2ba8220f81a4d6cb3611ca53db91fe1 P 8337af1943564c0dad18b45a597b805a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:56.132557 10592 tablet_bootstrap.cc:492] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a: Bootstrap starting.
I20240502 07:58:56.133280 10589 tablet_bootstrap.cc:492] T 50f059668c6b4205a15d2c076e2789b0 P 6cc5b9962ed24441bcba0c07b9f5438e: No bootstrap required, opened a new log
I20240502 07:58:56.133365 10589 ts_tablet_manager.cc:1367] T 50f059668c6b4205a15d2c076e2789b0 P 6cc5b9962ed24441bcba0c07b9f5438e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 07:58:56.133790 10589 raft_consensus.cc:348] T 50f059668c6b4205a15d2c076e2789b0 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:56.133895 10589 raft_consensus.cc:374] T 50f059668c6b4205a15d2c076e2789b0 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:56.133917 10589 raft_consensus.cc:729] T 50f059668c6b4205a15d2c076e2789b0 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cc5b9962ed24441bcba0c07b9f5438e, State: Initialized, Role: FOLLOWER
I20240502 07:58:56.134032 10589 consensus_queue.cc:260] T 50f059668c6b4205a15d2c076e2789b0 P 6cc5b9962ed24441bcba0c07b9f5438e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:56.134284 10589 ts_tablet_manager.cc:1398] T 50f059668c6b4205a15d2c076e2789b0 P 6cc5b9962ed24441bcba0c07b9f5438e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:56.134431 10592 tablet_bootstrap.cc:654] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:56.135499 10592 tablet_bootstrap.cc:492] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a: No bootstrap required, opened a new log
I20240502 07:58:56.135586 10592 ts_tablet_manager.cc:1367] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 07:58:56.135927 10592 raft_consensus.cc:348] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:56.136014 10592 raft_consensus.cc:374] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:56.136044 10592 raft_consensus.cc:729] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8337af1943564c0dad18b45a597b805a, State: Initialized, Role: FOLLOWER
I20240502 07:58:56.136132 10592 consensus_queue.cc:260] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:56.136363 10592 ts_tablet_manager.cc:1398] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:56.181628 10575 raft_consensus.cc:482] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:56.181762 10575 raft_consensus.cc:504] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:56.182246 10575 leader_election.cc:288] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191), 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379)
I20240502 07:58:56.183104 9907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d562a250cee247a48e573b25ecf00429" candidate_uuid: "8337af1943564c0dad18b45a597b805a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" is_pre_election: true
I20240502 07:58:56.183276 9907 raft_consensus.cc:2454] T d562a250cee247a48e573b25ecf00429 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8337af1943564c0dad18b45a597b805a in term 0.
I20240502 07:58:56.183576 9580 leader_election.cc:302] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cc5b9962ed24441bcba0c07b9f5438e, 8337af1943564c0dad18b45a597b805a; no voters:
I20240502 07:58:56.183758 10575 raft_consensus.cc:2781] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:56.183817 10575 raft_consensus.cc:482] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:56.183841 10575 raft_consensus.cc:3037] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:56.184628 10575 raft_consensus.cc:504] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:56.184950 10575 leader_election.cc:288] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [CANDIDATE]: Term 1 election: Requested vote from peers 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191), 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379)
I20240502 07:58:56.185307 9907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d562a250cee247a48e573b25ecf00429" candidate_uuid: "8337af1943564c0dad18b45a597b805a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cc5b9962ed24441bcba0c07b9f5438e"
I20240502 07:58:56.185413 9907 raft_consensus.cc:3037] T d562a250cee247a48e573b25ecf00429 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:56.185746 9777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d562a250cee247a48e573b25ecf00429" candidate_uuid: "8337af1943564c0dad18b45a597b805a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1472c64688914149a318c7b8850a3df3" is_pre_election: true
I20240502 07:58:56.185773 9776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d562a250cee247a48e573b25ecf00429" candidate_uuid: "8337af1943564c0dad18b45a597b805a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1472c64688914149a318c7b8850a3df3"
I20240502 07:58:56.185945 9776 raft_consensus.cc:3037] T d562a250cee247a48e573b25ecf00429 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:56.186367 9907 raft_consensus.cc:2454] T d562a250cee247a48e573b25ecf00429 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8337af1943564c0dad18b45a597b805a in term 1.
I20240502 07:58:56.186643 9580 leader_election.cc:302] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cc5b9962ed24441bcba0c07b9f5438e, 8337af1943564c0dad18b45a597b805a; no voters:
I20240502 07:58:56.186836 10575 raft_consensus.cc:2781] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:56.186887 9776 raft_consensus.cc:2454] T d562a250cee247a48e573b25ecf00429 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8337af1943564c0dad18b45a597b805a in term 1.
I20240502 07:58:56.186923 10575 raft_consensus.cc:686] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Becoming Leader. State: Replica: 8337af1943564c0dad18b45a597b805a, State: Running, Role: LEADER
I20240502 07:58:56.187073 10575 consensus_queue.cc:237] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:56.188706 9371 catalog_manager.cc:5484] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a reported cstate change: term changed from 0 to 1, leader changed from <none> to 8337af1943564c0dad18b45a597b805a (127.3.139.66). New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: HEALTHY } } }
I20240502 07:58:56.237707 10575 raft_consensus.cc:482] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:56.237854 10575 raft_consensus.cc:504] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:56.238327 10575 leader_election.cc:288] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379), 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191)
I20240502 07:58:56.238798 9907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50f059668c6b4205a15d2c076e2789b0" candidate_uuid: "8337af1943564c0dad18b45a597b805a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" is_pre_election: true
I20240502 07:58:56.238798 9776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50f059668c6b4205a15d2c076e2789b0" candidate_uuid: "8337af1943564c0dad18b45a597b805a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1472c64688914149a318c7b8850a3df3" is_pre_election: true
I20240502 07:58:56.238952 9907 raft_consensus.cc:2454] T 50f059668c6b4205a15d2c076e2789b0 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8337af1943564c0dad18b45a597b805a in term 0.
I20240502 07:58:56.238952 9776 raft_consensus.cc:2454] T 50f059668c6b4205a15d2c076e2789b0 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8337af1943564c0dad18b45a597b805a in term 0.
I20240502 07:58:56.239253 9580 leader_election.cc:302] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cc5b9962ed24441bcba0c07b9f5438e, 8337af1943564c0dad18b45a597b805a; no voters:
I20240502 07:58:56.239426 10575 raft_consensus.cc:2781] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:56.239490 10575 raft_consensus.cc:482] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:56.239516 10575 raft_consensus.cc:3037] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:56.240350 10575 raft_consensus.cc:504] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:56.240725 10575 leader_election.cc:288] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a [CANDIDATE]: Term 1 election: Requested vote from peers 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379), 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191)
I20240502 07:58:56.241008 9907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50f059668c6b4205a15d2c076e2789b0" candidate_uuid: "8337af1943564c0dad18b45a597b805a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cc5b9962ed24441bcba0c07b9f5438e"
I20240502 07:58:56.241043 9776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50f059668c6b4205a15d2c076e2789b0" candidate_uuid: "8337af1943564c0dad18b45a597b805a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1472c64688914149a318c7b8850a3df3"
I20240502 07:58:56.241124 9907 raft_consensus.cc:3037] T 50f059668c6b4205a15d2c076e2789b0 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:56.241135 9776 raft_consensus.cc:3037] T 50f059668c6b4205a15d2c076e2789b0 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:56.241811 9776 raft_consensus.cc:2454] T 50f059668c6b4205a15d2c076e2789b0 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8337af1943564c0dad18b45a597b805a in term 1.
I20240502 07:58:56.242017 9907 raft_consensus.cc:2454] T 50f059668c6b4205a15d2c076e2789b0 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8337af1943564c0dad18b45a597b805a in term 1.
I20240502 07:58:56.242079 9582 leader_election.cc:302] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1472c64688914149a318c7b8850a3df3, 8337af1943564c0dad18b45a597b805a; no voters:
I20240502 07:58:56.242235 10575 raft_consensus.cc:2781] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:56.242352 10575 raft_consensus.cc:686] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Becoming Leader. State: Replica: 8337af1943564c0dad18b45a597b805a, State: Running, Role: LEADER
I20240502 07:58:56.242487 10575 consensus_queue.cc:237] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:56.244058 9371 catalog_manager.cc:5484] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a reported cstate change: term changed from 0 to 1, leader changed from <none> to 8337af1943564c0dad18b45a597b805a (127.3.139.66). New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:56.244560 10508 raft_consensus.cc:482] T e2ba8220f81a4d6cb3611ca53db91fe1 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:56.244660 10508 raft_consensus.cc:504] T e2ba8220f81a4d6cb3611ca53db91fe1 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:56.245839 10508 leader_election.cc:288] T e2ba8220f81a4d6cb3611ca53db91fe1 P a201db721bda4dc3a4422dc8fa9bf72e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667), 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191)
I20240502 07:58:56.249565 9776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2ba8220f81a4d6cb3611ca53db91fe1" candidate_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1472c64688914149a318c7b8850a3df3" is_pre_election: true
I20240502 07:58:56.249682 9776 raft_consensus.cc:2454] T e2ba8220f81a4d6cb3611ca53db91fe1 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a201db721bda4dc3a4422dc8fa9bf72e in term 0.
I20240502 07:58:56.249923 9645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2ba8220f81a4d6cb3611ca53db91fe1" candidate_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8337af1943564c0dad18b45a597b805a" is_pre_election: true
I20240502 07:58:56.250041 9645 raft_consensus.cc:2454] T e2ba8220f81a4d6cb3611ca53db91fe1 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a201db721bda4dc3a4422dc8fa9bf72e in term 0.
I20240502 07:58:56.250285 9451 leader_election.cc:302] T e2ba8220f81a4d6cb3611ca53db91fe1 P a201db721bda4dc3a4422dc8fa9bf72e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1472c64688914149a318c7b8850a3df3, a201db721bda4dc3a4422dc8fa9bf72e; no voters:
I20240502 07:58:56.250497 10508 raft_consensus.cc:2781] T e2ba8220f81a4d6cb3611ca53db91fe1 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:56.250551 10508 raft_consensus.cc:482] T e2ba8220f81a4d6cb3611ca53db91fe1 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:56.250576 10508 raft_consensus.cc:3037] T e2ba8220f81a4d6cb3611ca53db91fe1 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:56.251358 10508 raft_consensus.cc:504] T e2ba8220f81a4d6cb3611ca53db91fe1 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:56.251646 10508 leader_election.cc:288] T e2ba8220f81a4d6cb3611ca53db91fe1 P a201db721bda4dc3a4422dc8fa9bf72e [CANDIDATE]: Term 1 election: Requested vote from peers 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667), 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191)
I20240502 07:58:56.251921 9645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2ba8220f81a4d6cb3611ca53db91fe1" candidate_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8337af1943564c0dad18b45a597b805a"
I20240502 07:58:56.251935 9776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2ba8220f81a4d6cb3611ca53db91fe1" candidate_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1472c64688914149a318c7b8850a3df3"
I20240502 07:58:56.252009 9645 raft_consensus.cc:3037] T e2ba8220f81a4d6cb3611ca53db91fe1 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:56.252034 9776 raft_consensus.cc:3037] T e2ba8220f81a4d6cb3611ca53db91fe1 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:56.252686 9776 raft_consensus.cc:2454] T e2ba8220f81a4d6cb3611ca53db91fe1 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a201db721bda4dc3a4422dc8fa9bf72e in term 1.
I20240502 07:58:56.252689 9645 raft_consensus.cc:2454] T e2ba8220f81a4d6cb3611ca53db91fe1 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a201db721bda4dc3a4422dc8fa9bf72e in term 1.
I20240502 07:58:56.252945 9449 leader_election.cc:302] T e2ba8220f81a4d6cb3611ca53db91fe1 P a201db721bda4dc3a4422dc8fa9bf72e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8337af1943564c0dad18b45a597b805a, a201db721bda4dc3a4422dc8fa9bf72e; no voters:
I20240502 07:58:56.253094 10508 raft_consensus.cc:2781] T e2ba8220f81a4d6cb3611ca53db91fe1 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:56.253170 10508 raft_consensus.cc:686] T e2ba8220f81a4d6cb3611ca53db91fe1 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Becoming Leader. State: Replica: a201db721bda4dc3a4422dc8fa9bf72e, State: Running, Role: LEADER
I20240502 07:58:56.253443 10508 consensus_queue.cc:237] T e2ba8220f81a4d6cb3611ca53db91fe1 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:56.255288 9371 catalog_manager.cc:5484] T e2ba8220f81a4d6cb3611ca53db91fe1 P a201db721bda4dc3a4422dc8fa9bf72e reported cstate change: term changed from 0 to 1, leader changed from <none> to a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65). New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: UNKNOWN } } }
W20240502 07:58:56.379359 10606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:56.379593 10606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:56.382589 10606 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_1'
I20240502 07:58:56.451315 9776 raft_consensus.cc:1261] T 50f059668c6b4205a15d2c076e2789b0 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:56.451841 10513 consensus_queue.cc:1035] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:56.452000 9907 raft_consensus.cc:1261] T d562a250cee247a48e573b25ecf00429 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:56.452601 10513 consensus_queue.cc:1035] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:56.453011 9776 raft_consensus.cc:1261] T d562a250cee247a48e573b25ecf00429 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:56.453642 10513 consensus_queue.cc:1035] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:56.454280 9645 raft_consensus.cc:1261] T e2ba8220f81a4d6cb3611ca53db91fe1 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:56.454366 9776 raft_consensus.cc:1261] T e2ba8220f81a4d6cb3611ca53db91fe1 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:56.454766 10508 consensus_queue.cc:1035] T e2ba8220f81a4d6cb3611ca53db91fe1 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:56.458649 9908 raft_consensus.cc:1261] T 50f059668c6b4205a15d2c076e2789b0 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:56.459098 10513 consensus_queue.cc:1035] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:56.469623 10511 consensus_queue.cc:1035] T e2ba8220f81a4d6cb3611ca53db91fe1 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 92.3804 ms
time per row: 0.0461902 ms
I20240502 07:58:56.512468 9371 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47034:
name: "rebalance_test_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:58:56.517412 9626 tablet_service.cc:1467] Processing CreateTablet for tablet eddeb6a23ec1418eb35b414f5d52e7b7 (DEFAULT_TABLE table=rebalance_test_table_2 [id=1b8f60b832d543568624eb22602bb97c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:56.517427 9493 tablet_service.cc:1467] Processing CreateTablet for tablet eddeb6a23ec1418eb35b414f5d52e7b7 (DEFAULT_TABLE table=rebalance_test_table_2 [id=1b8f60b832d543568624eb22602bb97c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:56.517690 9626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eddeb6a23ec1418eb35b414f5d52e7b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:56.517689 9493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eddeb6a23ec1418eb35b414f5d52e7b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:56.519340 9757 tablet_service.cc:1467] Processing CreateTablet for tablet 16c2ef3c0f5c44b39327175f6519fb95 (DEFAULT_TABLE table=rebalance_test_table_2 [id=1b8f60b832d543568624eb22602bb97c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:56.519606 9757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16c2ef3c0f5c44b39327175f6519fb95. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:56.519702 10592 tablet_bootstrap.cc:492] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8337af1943564c0dad18b45a597b805a: Bootstrap starting.
I20240502 07:58:56.520504 10592 tablet_bootstrap.cc:654] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8337af1943564c0dad18b45a597b805a: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:56.521517 10588 tablet_bootstrap.cc:492] T 16c2ef3c0f5c44b39327175f6519fb95 P 1472c64688914149a318c7b8850a3df3: Bootstrap starting.
I20240502 07:58:56.521581 10592 tablet_bootstrap.cc:492] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8337af1943564c0dad18b45a597b805a: No bootstrap required, opened a new log
I20240502 07:58:56.521662 10592 ts_tablet_manager.cc:1367] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8337af1943564c0dad18b45a597b805a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240502 07:58:56.522081 10592 raft_consensus.cc:348] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:56.522186 10592 raft_consensus.cc:374] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:56.522215 10592 raft_consensus.cc:729] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8337af1943564c0dad18b45a597b805a, State: Initialized, Role: FOLLOWER
I20240502 07:58:56.522236 9494 tablet_service.cc:1467] Processing CreateTablet for tablet 16c2ef3c0f5c44b39327175f6519fb95 (DEFAULT_TABLE table=rebalance_test_table_2 [id=1b8f60b832d543568624eb22602bb97c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:56.522330 10588 tablet_bootstrap.cc:654] T 16c2ef3c0f5c44b39327175f6519fb95 P 1472c64688914149a318c7b8850a3df3: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:56.522323 10592 consensus_queue.cc:260] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8337af1943564c0dad18b45a597b805a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:56.522465 9494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16c2ef3c0f5c44b39327175f6519fb95. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:56.522567 10592 ts_tablet_manager.cc:1398] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8337af1943564c0dad18b45a597b805a: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240502 07:58:56.523275 10588 tablet_bootstrap.cc:492] T 16c2ef3c0f5c44b39327175f6519fb95 P 1472c64688914149a318c7b8850a3df3: No bootstrap required, opened a new log
I20240502 07:58:56.523365 10588 ts_tablet_manager.cc:1367] T 16c2ef3c0f5c44b39327175f6519fb95 P 1472c64688914149a318c7b8850a3df3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:56.523689 10588 raft_consensus.cc:348] T 16c2ef3c0f5c44b39327175f6519fb95 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:56.523773 10588 raft_consensus.cc:374] T 16c2ef3c0f5c44b39327175f6519fb95 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:56.523800 10588 raft_consensus.cc:729] T 16c2ef3c0f5c44b39327175f6519fb95 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1472c64688914149a318c7b8850a3df3, State: Initialized, Role: FOLLOWER
I20240502 07:58:56.523918 10588 consensus_queue.cc:260] T 16c2ef3c0f5c44b39327175f6519fb95 P 1472c64688914149a318c7b8850a3df3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:56.524080 10590 tablet_bootstrap.cc:492] T 16c2ef3c0f5c44b39327175f6519fb95 P a201db721bda4dc3a4422dc8fa9bf72e: Bootstrap starting.
I20240502 07:58:56.524145 10588 ts_tablet_manager.cc:1398] T 16c2ef3c0f5c44b39327175f6519fb95 P 1472c64688914149a318c7b8850a3df3: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:56.524248 9495 tablet_service.cc:1467] Processing CreateTablet for tablet 3255d89d51a94426a78364230c9b249b (DEFAULT_TABLE table=rebalance_test_table_2 [id=1b8f60b832d543568624eb22602bb97c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:56.524459 9495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3255d89d51a94426a78364230c9b249b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:56.524878 10590 tablet_bootstrap.cc:654] T 16c2ef3c0f5c44b39327175f6519fb95 P a201db721bda4dc3a4422dc8fa9bf72e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:56.526930 9626 tablet_service.cc:1467] Processing CreateTablet for tablet 3255d89d51a94426a78364230c9b249b (DEFAULT_TABLE table=rebalance_test_table_2 [id=1b8f60b832d543568624eb22602bb97c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:56.527179 9626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3255d89d51a94426a78364230c9b249b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:56.527170 9888 tablet_service.cc:1467] Processing CreateTablet for tablet eddeb6a23ec1418eb35b414f5d52e7b7 (DEFAULT_TABLE table=rebalance_test_table_2 [id=1b8f60b832d543568624eb22602bb97c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:56.527419 9887 tablet_service.cc:1467] Processing CreateTablet for tablet 16c2ef3c0f5c44b39327175f6519fb95 (DEFAULT_TABLE table=rebalance_test_table_2 [id=1b8f60b832d543568624eb22602bb97c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:56.527420 9888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eddeb6a23ec1418eb35b414f5d52e7b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:56.527642 9887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16c2ef3c0f5c44b39327175f6519fb95. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:56.528775 10592 tablet_bootstrap.cc:492] T 3255d89d51a94426a78364230c9b249b P 8337af1943564c0dad18b45a597b805a: Bootstrap starting.
I20240502 07:58:56.528738 9886 tablet_service.cc:1467] Processing CreateTablet for tablet 3255d89d51a94426a78364230c9b249b (DEFAULT_TABLE table=rebalance_test_table_2 [id=1b8f60b832d543568624eb22602bb97c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:56.528954 9886 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3255d89d51a94426a78364230c9b249b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:56.529208 10590 tablet_bootstrap.cc:492] T 16c2ef3c0f5c44b39327175f6519fb95 P a201db721bda4dc3a4422dc8fa9bf72e: No bootstrap required, opened a new log
I20240502 07:58:56.529280 10590 ts_tablet_manager.cc:1367] T 16c2ef3c0f5c44b39327175f6519fb95 P a201db721bda4dc3a4422dc8fa9bf72e: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 07:58:56.529469 10589 tablet_bootstrap.cc:492] T eddeb6a23ec1418eb35b414f5d52e7b7 P 6cc5b9962ed24441bcba0c07b9f5438e: Bootstrap starting.
I20240502 07:58:56.529629 10592 tablet_bootstrap.cc:654] T 3255d89d51a94426a78364230c9b249b P 8337af1943564c0dad18b45a597b805a: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:56.529984 10590 raft_consensus.cc:348] T 16c2ef3c0f5c44b39327175f6519fb95 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:56.530093 10590 raft_consensus.cc:374] T 16c2ef3c0f5c44b39327175f6519fb95 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:56.530122 10590 raft_consensus.cc:729] T 16c2ef3c0f5c44b39327175f6519fb95 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a201db721bda4dc3a4422dc8fa9bf72e, State: Initialized, Role: FOLLOWER
I20240502 07:58:56.530232 10590 consensus_queue.cc:260] T 16c2ef3c0f5c44b39327175f6519fb95 P a201db721bda4dc3a4422dc8fa9bf72e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:56.530290 10589 tablet_bootstrap.cc:654] T eddeb6a23ec1418eb35b414f5d52e7b7 P 6cc5b9962ed24441bcba0c07b9f5438e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:56.530480 10590 ts_tablet_manager.cc:1398] T 16c2ef3c0f5c44b39327175f6519fb95 P a201db721bda4dc3a4422dc8fa9bf72e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:56.530598 10590 tablet_bootstrap.cc:492] T 3255d89d51a94426a78364230c9b249b P a201db721bda4dc3a4422dc8fa9bf72e: Bootstrap starting.
I20240502 07:58:56.531369 10590 tablet_bootstrap.cc:654] T 3255d89d51a94426a78364230c9b249b P a201db721bda4dc3a4422dc8fa9bf72e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:56.531889 10589 tablet_bootstrap.cc:492] T eddeb6a23ec1418eb35b414f5d52e7b7 P 6cc5b9962ed24441bcba0c07b9f5438e: No bootstrap required, opened a new log
I20240502 07:58:56.531966 10589 ts_tablet_manager.cc:1367] T eddeb6a23ec1418eb35b414f5d52e7b7 P 6cc5b9962ed24441bcba0c07b9f5438e: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 07:58:56.532383 10590 tablet_bootstrap.cc:492] T 3255d89d51a94426a78364230c9b249b P a201db721bda4dc3a4422dc8fa9bf72e: No bootstrap required, opened a new log
I20240502 07:58:56.532353 10589 raft_consensus.cc:348] T eddeb6a23ec1418eb35b414f5d52e7b7 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:56.532444 10589 raft_consensus.cc:374] T eddeb6a23ec1418eb35b414f5d52e7b7 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:56.532450 10590 ts_tablet_manager.cc:1367] T 3255d89d51a94426a78364230c9b249b P a201db721bda4dc3a4422dc8fa9bf72e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:56.532475 10589 raft_consensus.cc:729] T eddeb6a23ec1418eb35b414f5d52e7b7 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cc5b9962ed24441bcba0c07b9f5438e, State: Initialized, Role: FOLLOWER
I20240502 07:58:56.532583 10589 consensus_queue.cc:260] T eddeb6a23ec1418eb35b414f5d52e7b7 P 6cc5b9962ed24441bcba0c07b9f5438e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:56.532829 10589 ts_tablet_manager.cc:1398] T eddeb6a23ec1418eb35b414f5d52e7b7 P 6cc5b9962ed24441bcba0c07b9f5438e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:56.532817 10590 raft_consensus.cc:348] T 3255d89d51a94426a78364230c9b249b P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:56.532915 10590 raft_consensus.cc:374] T 3255d89d51a94426a78364230c9b249b P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:56.532945 10590 raft_consensus.cc:729] T 3255d89d51a94426a78364230c9b249b P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a201db721bda4dc3a4422dc8fa9bf72e, State: Initialized, Role: FOLLOWER
I20240502 07:58:56.532979 10589 tablet_bootstrap.cc:492] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e: Bootstrap starting.
I20240502 07:58:56.533073 10590 consensus_queue.cc:260] T 3255d89d51a94426a78364230c9b249b P a201db721bda4dc3a4422dc8fa9bf72e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:56.533407 10590 ts_tablet_manager.cc:1398] T 3255d89d51a94426a78364230c9b249b P a201db721bda4dc3a4422dc8fa9bf72e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:56.533555 10590 tablet_bootstrap.cc:492] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e: Bootstrap starting.
I20240502 07:58:56.533778 10589 tablet_bootstrap.cc:654] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:56.534343 10590 tablet_bootstrap.cc:654] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:56.535352 10590 tablet_bootstrap.cc:492] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e: No bootstrap required, opened a new log
I20240502 07:58:56.535437 10590 ts_tablet_manager.cc:1367] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:56.535765 10589 tablet_bootstrap.cc:492] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e: No bootstrap required, opened a new log
I20240502 07:58:56.535830 10592 tablet_bootstrap.cc:492] T 3255d89d51a94426a78364230c9b249b P 8337af1943564c0dad18b45a597b805a: No bootstrap required, opened a new log
I20240502 07:58:56.535812 10590 raft_consensus.cc:348] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:56.535900 10590 raft_consensus.cc:374] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:56.535907 10592 ts_tablet_manager.cc:1367] T 3255d89d51a94426a78364230c9b249b P 8337af1943564c0dad18b45a597b805a: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.001s
I20240502 07:58:56.535920 10589 ts_tablet_manager.cc:1367] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 07:58:56.535929 10590 raft_consensus.cc:729] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a201db721bda4dc3a4422dc8fa9bf72e, State: Initialized, Role: FOLLOWER
I20240502 07:58:56.536026 10590 consensus_queue.cc:260] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:56.536232 10590 ts_tablet_manager.cc:1398] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:56.536294 10592 raft_consensus.cc:348] T 3255d89d51a94426a78364230c9b249b P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:56.536412 10592 raft_consensus.cc:374] T 3255d89d51a94426a78364230c9b249b P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:56.536454 10592 raft_consensus.cc:729] T 3255d89d51a94426a78364230c9b249b P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8337af1943564c0dad18b45a597b805a, State: Initialized, Role: FOLLOWER
I20240502 07:58:56.536569 10592 consensus_queue.cc:260] T 3255d89d51a94426a78364230c9b249b P 8337af1943564c0dad18b45a597b805a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:56.536293 10589 raft_consensus.cc:348] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:56.536701 10589 raft_consensus.cc:374] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:56.536729 10589 raft_consensus.cc:729] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cc5b9962ed24441bcba0c07b9f5438e, State: Initialized, Role: FOLLOWER
I20240502 07:58:56.536857 10592 ts_tablet_manager.cc:1398] T 3255d89d51a94426a78364230c9b249b P 8337af1943564c0dad18b45a597b805a: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240502 07:58:56.536836 10589 consensus_queue.cc:260] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:56.537073 10589 ts_tablet_manager.cc:1398] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:56.537247 10589 tablet_bootstrap.cc:492] T 16c2ef3c0f5c44b39327175f6519fb95 P 6cc5b9962ed24441bcba0c07b9f5438e: Bootstrap starting.
I20240502 07:58:56.538901 10589 tablet_bootstrap.cc:654] T 16c2ef3c0f5c44b39327175f6519fb95 P 6cc5b9962ed24441bcba0c07b9f5438e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:56.539860 10589 tablet_bootstrap.cc:492] T 16c2ef3c0f5c44b39327175f6519fb95 P 6cc5b9962ed24441bcba0c07b9f5438e: No bootstrap required, opened a new log
I20240502 07:58:56.539924 10589 ts_tablet_manager.cc:1367] T 16c2ef3c0f5c44b39327175f6519fb95 P 6cc5b9962ed24441bcba0c07b9f5438e: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240502 07:58:56.540211 10589 raft_consensus.cc:348] T 16c2ef3c0f5c44b39327175f6519fb95 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:56.540283 10589 raft_consensus.cc:374] T 16c2ef3c0f5c44b39327175f6519fb95 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:56.540333 10589 raft_consensus.cc:729] T 16c2ef3c0f5c44b39327175f6519fb95 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cc5b9962ed24441bcba0c07b9f5438e, State: Initialized, Role: FOLLOWER
I20240502 07:58:56.540439 10589 consensus_queue.cc:260] T 16c2ef3c0f5c44b39327175f6519fb95 P 6cc5b9962ed24441bcba0c07b9f5438e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:56.540654 10589 ts_tablet_manager.cc:1398] T 16c2ef3c0f5c44b39327175f6519fb95 P 6cc5b9962ed24441bcba0c07b9f5438e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:56.555676 10511 raft_consensus.cc:482] T 16c2ef3c0f5c44b39327175f6519fb95 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:56.555783 10511 raft_consensus.cc:504] T 16c2ef3c0f5c44b39327175f6519fb95 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:56.556195 10511 leader_election.cc:288] T 16c2ef3c0f5c44b39327175f6519fb95 P a201db721bda4dc3a4422dc8fa9bf72e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379), 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191)
I20240502 07:58:56.556581 9776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16c2ef3c0f5c44b39327175f6519fb95" candidate_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1472c64688914149a318c7b8850a3df3" is_pre_election: true
I20240502 07:58:56.556731 9776 raft_consensus.cc:2454] T 16c2ef3c0f5c44b39327175f6519fb95 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a201db721bda4dc3a4422dc8fa9bf72e in term 0.
I20240502 07:58:56.557013 9451 leader_election.cc:302] T 16c2ef3c0f5c44b39327175f6519fb95 P a201db721bda4dc3a4422dc8fa9bf72e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1472c64688914149a318c7b8850a3df3, a201db721bda4dc3a4422dc8fa9bf72e; no voters:
I20240502 07:58:56.557178 10511 raft_consensus.cc:2781] T 16c2ef3c0f5c44b39327175f6519fb95 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:56.557220 10511 raft_consensus.cc:482] T 16c2ef3c0f5c44b39327175f6519fb95 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:56.557251 10511 raft_consensus.cc:3037] T 16c2ef3c0f5c44b39327175f6519fb95 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:56.558243 10511 raft_consensus.cc:504] T 16c2ef3c0f5c44b39327175f6519fb95 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:56.558573 10511 leader_election.cc:288] T 16c2ef3c0f5c44b39327175f6519fb95 P a201db721bda4dc3a4422dc8fa9bf72e [CANDIDATE]: Term 1 election: Requested vote from peers 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379), 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191)
I20240502 07:58:56.558934 9776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16c2ef3c0f5c44b39327175f6519fb95" candidate_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1472c64688914149a318c7b8850a3df3"
I20240502 07:58:56.559043 9776 raft_consensus.cc:3037] T 16c2ef3c0f5c44b39327175f6519fb95 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:56.559911 9776 raft_consensus.cc:2454] T 16c2ef3c0f5c44b39327175f6519fb95 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a201db721bda4dc3a4422dc8fa9bf72e in term 1.
I20240502 07:58:56.560181 9451 leader_election.cc:302] T 16c2ef3c0f5c44b39327175f6519fb95 P a201db721bda4dc3a4422dc8fa9bf72e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1472c64688914149a318c7b8850a3df3, a201db721bda4dc3a4422dc8fa9bf72e; no voters:
I20240502 07:58:56.560353 10511 raft_consensus.cc:2781] T 16c2ef3c0f5c44b39327175f6519fb95 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:56.560281 9907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16c2ef3c0f5c44b39327175f6519fb95" candidate_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" is_pre_election: true
I20240502 07:58:56.560309 9908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16c2ef3c0f5c44b39327175f6519fb95" candidate_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cc5b9962ed24441bcba0c07b9f5438e"
I20240502 07:58:56.560433 10511 raft_consensus.cc:686] T 16c2ef3c0f5c44b39327175f6519fb95 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Becoming Leader. State: Replica: a201db721bda4dc3a4422dc8fa9bf72e, State: Running, Role: LEADER
I20240502 07:58:56.560431 9907 raft_consensus.cc:2454] T 16c2ef3c0f5c44b39327175f6519fb95 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a201db721bda4dc3a4422dc8fa9bf72e in term 0.
I20240502 07:58:56.560549 10511 consensus_queue.cc:237] T 16c2ef3c0f5c44b39327175f6519fb95 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:56.561898 9373 catalog_manager.cc:5484] T 16c2ef3c0f5c44b39327175f6519fb95 P a201db721bda4dc3a4422dc8fa9bf72e reported cstate change: term changed from 0 to 1, leader changed from <none> to a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65). New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:56.638309 10514 raft_consensus.cc:482] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:56.638432 10514 raft_consensus.cc:504] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:56.639405 10514 leader_election.cc:288] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551), 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667)
I20240502 07:58:56.642706 9645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3255d89d51a94426a78364230c9b249b" candidate_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8337af1943564c0dad18b45a597b805a" is_pre_election: true
I20240502 07:58:56.642869 9645 raft_consensus.cc:2454] T 3255d89d51a94426a78364230c9b249b P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cc5b9962ed24441bcba0c07b9f5438e in term 0.
I20240502 07:58:56.643010 9514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3255d89d51a94426a78364230c9b249b" candidate_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" is_pre_election: true
I20240502 07:58:56.643172 9514 raft_consensus.cc:2454] T 3255d89d51a94426a78364230c9b249b P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cc5b9962ed24441bcba0c07b9f5438e in term 0.
I20240502 07:58:56.643208 9842 leader_election.cc:302] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cc5b9962ed24441bcba0c07b9f5438e, 8337af1943564c0dad18b45a597b805a; no voters:
I20240502 07:58:56.643442 10514 raft_consensus.cc:2781] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:56.643502 10514 raft_consensus.cc:482] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:56.643530 10514 raft_consensus.cc:3037] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:56.644338 10514 raft_consensus.cc:504] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:56.644657 10514 leader_election.cc:288] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e [CANDIDATE]: Term 1 election: Requested vote from peers a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551), 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667)
I20240502 07:58:56.644904 9514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3255d89d51a94426a78364230c9b249b" candidate_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a201db721bda4dc3a4422dc8fa9bf72e"
I20240502 07:58:56.645015 9514 raft_consensus.cc:3037] T 3255d89d51a94426a78364230c9b249b P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:56.645052 9645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3255d89d51a94426a78364230c9b249b" candidate_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8337af1943564c0dad18b45a597b805a"
I20240502 07:58:56.645155 9645 raft_consensus.cc:3037] T 3255d89d51a94426a78364230c9b249b P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:56.645956 9514 raft_consensus.cc:2454] T 3255d89d51a94426a78364230c9b249b P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cc5b9962ed24441bcba0c07b9f5438e in term 1.
I20240502 07:58:56.646035 9645 raft_consensus.cc:2454] T 3255d89d51a94426a78364230c9b249b P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cc5b9962ed24441bcba0c07b9f5438e in term 1.
I20240502 07:58:56.646222 9844 leader_election.cc:302] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cc5b9962ed24441bcba0c07b9f5438e, a201db721bda4dc3a4422dc8fa9bf72e; no voters:
I20240502 07:58:56.646468 10514 raft_consensus.cc:2781] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:56.646628 10514 raft_consensus.cc:686] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 LEADER]: Becoming Leader. State: Replica: 6cc5b9962ed24441bcba0c07b9f5438e, State: Running, Role: LEADER
I20240502 07:58:56.646901 10514 consensus_queue.cc:237] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:56.649024 9373 catalog_manager.cc:5484] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68). New cstate: current_term: 1 leader_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:56.840991 10511 raft_consensus.cc:482] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:56.841120 10511 raft_consensus.cc:504] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:56.841504 10511 leader_election.cc:288] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667), 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379)
I20240502 07:58:56.841962 9645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eddeb6a23ec1418eb35b414f5d52e7b7" candidate_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8337af1943564c0dad18b45a597b805a" is_pre_election: true
I20240502 07:58:56.841977 9908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eddeb6a23ec1418eb35b414f5d52e7b7" candidate_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" is_pre_election: true
I20240502 07:58:56.842116 9645 raft_consensus.cc:2454] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a201db721bda4dc3a4422dc8fa9bf72e in term 0.
I20240502 07:58:56.842116 9908 raft_consensus.cc:2454] T eddeb6a23ec1418eb35b414f5d52e7b7 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a201db721bda4dc3a4422dc8fa9bf72e in term 0.
I20240502 07:58:56.842461 9449 leader_election.cc:302] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8337af1943564c0dad18b45a597b805a, a201db721bda4dc3a4422dc8fa9bf72e; no voters:
I20240502 07:58:56.842633 10511 raft_consensus.cc:2781] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:56.842710 10511 raft_consensus.cc:482] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:56.842756 10511 raft_consensus.cc:3037] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:56.843564 10511 raft_consensus.cc:504] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:56.843871 10511 leader_election.cc:288] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [CANDIDATE]: Term 1 election: Requested vote from peers 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667), 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379)
I20240502 07:58:56.844221 9645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eddeb6a23ec1418eb35b414f5d52e7b7" candidate_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8337af1943564c0dad18b45a597b805a"
I20240502 07:58:56.844270 9908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eddeb6a23ec1418eb35b414f5d52e7b7" candidate_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cc5b9962ed24441bcba0c07b9f5438e"
I20240502 07:58:56.844341 9645 raft_consensus.cc:3037] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:56.844400 9908 raft_consensus.cc:3037] T eddeb6a23ec1418eb35b414f5d52e7b7 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:56.845255 9908 raft_consensus.cc:2454] T eddeb6a23ec1418eb35b414f5d52e7b7 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a201db721bda4dc3a4422dc8fa9bf72e in term 1.
I20240502 07:58:56.845255 9645 raft_consensus.cc:2454] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a201db721bda4dc3a4422dc8fa9bf72e in term 1.
I20240502 07:58:56.845515 9449 leader_election.cc:302] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8337af1943564c0dad18b45a597b805a, a201db721bda4dc3a4422dc8fa9bf72e; no voters:
I20240502 07:58:56.845671 10511 raft_consensus.cc:2781] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:56.845762 10511 raft_consensus.cc:686] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Becoming Leader. State: Replica: a201db721bda4dc3a4422dc8fa9bf72e, State: Running, Role: LEADER
I20240502 07:58:56.845930 10511 consensus_queue.cc:237] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:56.847540 9373 catalog_manager.cc:5484] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e reported cstate change: term changed from 0 to 1, leader changed from <none> to a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65). New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: UNKNOWN } } }
W20240502 07:58:56.991804 10658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:56.992028 10658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:56.995033 10658 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_2'
I20240502 07:58:57.063083 9776 raft_consensus.cc:1261] T 16c2ef3c0f5c44b39327175f6519fb95 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:57.063525 10511 consensus_queue.cc:1035] T 16c2ef3c0f5c44b39327175f6519fb95 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:57.064105 9908 raft_consensus.cc:3037] T 16c2ef3c0f5c44b39327175f6519fb95 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:57.065111 9908 raft_consensus.cc:1261] T 16c2ef3c0f5c44b39327175f6519fb95 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:57.065600 10511 consensus_queue.cc:1035] T 16c2ef3c0f5c44b39327175f6519fb95 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:57.066236 9645 raft_consensus.cc:1261] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:57.066468 9646 raft_consensus.cc:1261] T 3255d89d51a94426a78364230c9b249b P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:57.067355 9514 raft_consensus.cc:1261] T 3255d89d51a94426a78364230c9b249b P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:57.068087 10511 consensus_queue.cc:1035] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:57.068466 10691 consensus_queue.cc:1035] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:57.069754 10655 consensus_queue.cc:1035] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:57.073453 9908 raft_consensus.cc:1261] T eddeb6a23ec1418eb35b414f5d52e7b7 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:57.074074 10511 consensus_queue.cc:1035] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 100.951 ms
time per row: 0.0504757 ms
I20240502 07:58:57.133668 9370 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47034:
name: "rebalance_test_table_3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:58:57.138653 9757 tablet_service.cc:1467] Processing CreateTablet for tablet d37b8a7f016647a7a03baedc37ed5419 (DEFAULT_TABLE table=rebalance_test_table_3 [id=47c27ad65ee1405898b667c2642e1b3a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:57.138949 9757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d37b8a7f016647a7a03baedc37ed5419. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.138950 9756 tablet_service.cc:1467] Processing CreateTablet for tablet a12a084dba23425996e9122c083c1444 (DEFAULT_TABLE table=rebalance_test_table_3 [id=47c27ad65ee1405898b667c2642e1b3a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:57.139261 9756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a12a084dba23425996e9122c083c1444. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.139360 9493 tablet_service.cc:1467] Processing CreateTablet for tablet d37b8a7f016647a7a03baedc37ed5419 (DEFAULT_TABLE table=rebalance_test_table_3 [id=47c27ad65ee1405898b667c2642e1b3a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:57.139525 9625 tablet_service.cc:1467] Processing CreateTablet for tablet 96dab19eeaea4648913045482b3e5e9e (DEFAULT_TABLE table=rebalance_test_table_3 [id=47c27ad65ee1405898b667c2642e1b3a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:57.139583 9493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d37b8a7f016647a7a03baedc37ed5419. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.139742 9625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96dab19eeaea4648913045482b3e5e9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.141058 9626 tablet_service.cc:1467] Processing CreateTablet for tablet d37b8a7f016647a7a03baedc37ed5419 (DEFAULT_TABLE table=rebalance_test_table_3 [id=47c27ad65ee1405898b667c2642e1b3a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:57.141261 9626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d37b8a7f016647a7a03baedc37ed5419. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.141566 10716 tablet_bootstrap.cc:492] T d37b8a7f016647a7a03baedc37ed5419 P a201db721bda4dc3a4422dc8fa9bf72e: Bootstrap starting.
I20240502 07:58:57.142036 10717 tablet_bootstrap.cc:492] T 96dab19eeaea4648913045482b3e5e9e P 8337af1943564c0dad18b45a597b805a: Bootstrap starting.
I20240502 07:58:57.142421 10716 tablet_bootstrap.cc:654] T d37b8a7f016647a7a03baedc37ed5419 P a201db721bda4dc3a4422dc8fa9bf72e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:57.142536 9756 tablet_service.cc:1467] Processing CreateTablet for tablet 96dab19eeaea4648913045482b3e5e9e (DEFAULT_TABLE table=rebalance_test_table_3 [id=47c27ad65ee1405898b667c2642e1b3a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:57.142732 9756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96dab19eeaea4648913045482b3e5e9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.142894 10717 tablet_bootstrap.cc:654] T 96dab19eeaea4648913045482b3e5e9e P 8337af1943564c0dad18b45a597b805a: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:57.143405 9887 tablet_service.cc:1467] Processing CreateTablet for tablet a12a084dba23425996e9122c083c1444 (DEFAULT_TABLE table=rebalance_test_table_3 [id=47c27ad65ee1405898b667c2642e1b3a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:57.143613 9887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a12a084dba23425996e9122c083c1444. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.143930 10715 tablet_bootstrap.cc:492] T a12a084dba23425996e9122c083c1444 P 1472c64688914149a318c7b8850a3df3: Bootstrap starting.
I20240502 07:58:57.144709 10715 tablet_bootstrap.cc:654] T a12a084dba23425996e9122c083c1444 P 1472c64688914149a318c7b8850a3df3: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:57.144834 9493 tablet_service.cc:1467] Processing CreateTablet for tablet a12a084dba23425996e9122c083c1444 (DEFAULT_TABLE table=rebalance_test_table_3 [id=47c27ad65ee1405898b667c2642e1b3a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:57.145046 9493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a12a084dba23425996e9122c083c1444. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.145542 9886 tablet_service.cc:1467] Processing CreateTablet for tablet 96dab19eeaea4648913045482b3e5e9e (DEFAULT_TABLE table=rebalance_test_table_3 [id=47c27ad65ee1405898b667c2642e1b3a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:57.145720 9886 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96dab19eeaea4648913045482b3e5e9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.145745 10715 tablet_bootstrap.cc:492] T a12a084dba23425996e9122c083c1444 P 1472c64688914149a318c7b8850a3df3: No bootstrap required, opened a new log
I20240502 07:58:57.145819 10715 ts_tablet_manager.cc:1367] T a12a084dba23425996e9122c083c1444 P 1472c64688914149a318c7b8850a3df3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:57.146190 10715 raft_consensus.cc:348] T a12a084dba23425996e9122c083c1444 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:58:57.146284 10715 raft_consensus.cc:374] T a12a084dba23425996e9122c083c1444 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:57.146310 10715 raft_consensus.cc:729] T a12a084dba23425996e9122c083c1444 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1472c64688914149a318c7b8850a3df3, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.146437 10715 consensus_queue.cc:260] T a12a084dba23425996e9122c083c1444 P 1472c64688914149a318c7b8850a3df3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:58:57.146703 10715 ts_tablet_manager.cc:1398] T a12a084dba23425996e9122c083c1444 P 1472c64688914149a318c7b8850a3df3: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:57.146863 10715 tablet_bootstrap.cc:492] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3: Bootstrap starting.
I20240502 07:58:57.147008 10721 tablet_bootstrap.cc:492] T a12a084dba23425996e9122c083c1444 P 6cc5b9962ed24441bcba0c07b9f5438e: Bootstrap starting.
I20240502 07:58:57.147599 10715 tablet_bootstrap.cc:654] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:57.147740 10721 tablet_bootstrap.cc:654] T a12a084dba23425996e9122c083c1444 P 6cc5b9962ed24441bcba0c07b9f5438e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:57.148347 10717 tablet_bootstrap.cc:492] T 96dab19eeaea4648913045482b3e5e9e P 8337af1943564c0dad18b45a597b805a: No bootstrap required, opened a new log
I20240502 07:58:57.148383 10716 tablet_bootstrap.cc:492] T d37b8a7f016647a7a03baedc37ed5419 P a201db721bda4dc3a4422dc8fa9bf72e: No bootstrap required, opened a new log
I20240502 07:58:57.148438 10717 ts_tablet_manager.cc:1367] T 96dab19eeaea4648913045482b3e5e9e P 8337af1943564c0dad18b45a597b805a: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240502 07:58:57.148455 10716 ts_tablet_manager.cc:1367] T d37b8a7f016647a7a03baedc37ed5419 P a201db721bda4dc3a4422dc8fa9bf72e: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 07:58:57.148867 10716 raft_consensus.cc:348] T d37b8a7f016647a7a03baedc37ed5419 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:57.148864 10717 raft_consensus.cc:348] T 96dab19eeaea4648913045482b3e5e9e P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:57.148960 10716 raft_consensus.cc:374] T d37b8a7f016647a7a03baedc37ed5419 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:57.148965 10717 raft_consensus.cc:374] T 96dab19eeaea4648913045482b3e5e9e P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:57.148993 10716 raft_consensus.cc:729] T d37b8a7f016647a7a03baedc37ed5419 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a201db721bda4dc3a4422dc8fa9bf72e, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.148994 10717 raft_consensus.cc:729] T 96dab19eeaea4648913045482b3e5e9e P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8337af1943564c0dad18b45a597b805a, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.149102 10716 consensus_queue.cc:260] T d37b8a7f016647a7a03baedc37ed5419 P a201db721bda4dc3a4422dc8fa9bf72e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:57.149120 10717 consensus_queue.cc:260] T 96dab19eeaea4648913045482b3e5e9e P 8337af1943564c0dad18b45a597b805a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:57.149334 10716 ts_tablet_manager.cc:1398] T d37b8a7f016647a7a03baedc37ed5419 P a201db721bda4dc3a4422dc8fa9bf72e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:57.149379 10717 ts_tablet_manager.cc:1398] T 96dab19eeaea4648913045482b3e5e9e P 8337af1943564c0dad18b45a597b805a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:57.149482 10716 tablet_bootstrap.cc:492] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e: Bootstrap starting.
I20240502 07:58:57.149534 10717 tablet_bootstrap.cc:492] T d37b8a7f016647a7a03baedc37ed5419 P 8337af1943564c0dad18b45a597b805a: Bootstrap starting.
I20240502 07:58:57.150135 10716 tablet_bootstrap.cc:654] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:57.150238 10717 tablet_bootstrap.cc:654] T d37b8a7f016647a7a03baedc37ed5419 P 8337af1943564c0dad18b45a597b805a: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:57.151125 10715 tablet_bootstrap.cc:492] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3: No bootstrap required, opened a new log
I20240502 07:58:57.151202 10715 ts_tablet_manager.cc:1367] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:58:57.151566 10715 raft_consensus.cc:348] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:57.151659 10715 raft_consensus.cc:374] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:57.151692 10715 raft_consensus.cc:729] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1472c64688914149a318c7b8850a3df3, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.152066 10717 tablet_bootstrap.cc:492] T d37b8a7f016647a7a03baedc37ed5419 P 8337af1943564c0dad18b45a597b805a: No bootstrap required, opened a new log
I20240502 07:58:57.152113 10716 tablet_bootstrap.cc:492] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e: No bootstrap required, opened a new log
I20240502 07:58:57.152138 10717 ts_tablet_manager.cc:1367] T d37b8a7f016647a7a03baedc37ed5419 P 8337af1943564c0dad18b45a597b805a: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:58:57.152186 10716 ts_tablet_manager.cc:1367] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:58:57.152527 10717 raft_consensus.cc:348] T d37b8a7f016647a7a03baedc37ed5419 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:57.152614 10717 raft_consensus.cc:374] T d37b8a7f016647a7a03baedc37ed5419 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:57.152592 10716 raft_consensus.cc:348] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:58:57.152647 10717 raft_consensus.cc:729] T d37b8a7f016647a7a03baedc37ed5419 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8337af1943564c0dad18b45a597b805a, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.152678 10716 raft_consensus.cc:374] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:57.152709 10716 raft_consensus.cc:729] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a201db721bda4dc3a4422dc8fa9bf72e, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.152773 10717 consensus_queue.cc:260] T d37b8a7f016647a7a03baedc37ed5419 P 8337af1943564c0dad18b45a597b805a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:57.152817 10716 consensus_queue.cc:260] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:58:57.152958 10715 consensus_queue.cc:260] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:57.153054 10717 ts_tablet_manager.cc:1398] T d37b8a7f016647a7a03baedc37ed5419 P 8337af1943564c0dad18b45a597b805a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:57.153218 10715 ts_tablet_manager.cc:1398] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:57.153396 10715 tablet_bootstrap.cc:492] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3: Bootstrap starting.
I20240502 07:58:57.154068 10715 tablet_bootstrap.cc:654] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:57.154747 10716 ts_tablet_manager.cc:1398] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:58:57.155041 10715 tablet_bootstrap.cc:492] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3: No bootstrap required, opened a new log
I20240502 07:58:57.155164 10715 ts_tablet_manager.cc:1367] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:57.155543 10715 raft_consensus.cc:348] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:57.155640 10715 raft_consensus.cc:374] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:57.155676 10715 raft_consensus.cc:729] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1472c64688914149a318c7b8850a3df3, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.155786 10715 consensus_queue.cc:260] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:57.156049 10715 ts_tablet_manager.cc:1398] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:57.156116 10721 tablet_bootstrap.cc:492] T a12a084dba23425996e9122c083c1444 P 6cc5b9962ed24441bcba0c07b9f5438e: No bootstrap required, opened a new log
I20240502 07:58:57.156193 10721 ts_tablet_manager.cc:1367] T a12a084dba23425996e9122c083c1444 P 6cc5b9962ed24441bcba0c07b9f5438e: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20240502 07:58:57.156628 10721 raft_consensus.cc:348] T a12a084dba23425996e9122c083c1444 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:58:57.156769 10721 raft_consensus.cc:374] T a12a084dba23425996e9122c083c1444 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:57.156841 10721 raft_consensus.cc:729] T a12a084dba23425996e9122c083c1444 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cc5b9962ed24441bcba0c07b9f5438e, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.157002 10721 consensus_queue.cc:260] T a12a084dba23425996e9122c083c1444 P 6cc5b9962ed24441bcba0c07b9f5438e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:58:57.157356 10721 ts_tablet_manager.cc:1398] T a12a084dba23425996e9122c083c1444 P 6cc5b9962ed24441bcba0c07b9f5438e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:57.157670 10721 tablet_bootstrap.cc:492] T 96dab19eeaea4648913045482b3e5e9e P 6cc5b9962ed24441bcba0c07b9f5438e: Bootstrap starting.
I20240502 07:58:57.159032 10721 tablet_bootstrap.cc:654] T 96dab19eeaea4648913045482b3e5e9e P 6cc5b9962ed24441bcba0c07b9f5438e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:57.160185 10721 tablet_bootstrap.cc:492] T 96dab19eeaea4648913045482b3e5e9e P 6cc5b9962ed24441bcba0c07b9f5438e: No bootstrap required, opened a new log
I20240502 07:58:57.160259 10721 ts_tablet_manager.cc:1367] T 96dab19eeaea4648913045482b3e5e9e P 6cc5b9962ed24441bcba0c07b9f5438e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:58:57.160621 10721 raft_consensus.cc:348] T 96dab19eeaea4648913045482b3e5e9e P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:57.160696 10721 raft_consensus.cc:374] T 96dab19eeaea4648913045482b3e5e9e P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:57.160723 10721 raft_consensus.cc:729] T 96dab19eeaea4648913045482b3e5e9e P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cc5b9962ed24441bcba0c07b9f5438e, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.160828 10721 consensus_queue.cc:260] T 96dab19eeaea4648913045482b3e5e9e P 6cc5b9962ed24441bcba0c07b9f5438e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:57.161018 10721 ts_tablet_manager.cc:1398] T 96dab19eeaea4648913045482b3e5e9e P 6cc5b9962ed24441bcba0c07b9f5438e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:57.172083 10558 raft_consensus.cc:482] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:57.172178 10558 raft_consensus.cc:504] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:57.172533 10558 leader_election.cc:288] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551), 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667)
I20240502 07:58:57.172856 9645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d37b8a7f016647a7a03baedc37ed5419" candidate_uuid: "1472c64688914149a318c7b8850a3df3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8337af1943564c0dad18b45a597b805a" is_pre_election: true
I20240502 07:58:57.172849 9514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d37b8a7f016647a7a03baedc37ed5419" candidate_uuid: "1472c64688914149a318c7b8850a3df3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" is_pre_election: true
I20240502 07:58:57.172983 9514 raft_consensus.cc:2454] T d37b8a7f016647a7a03baedc37ed5419 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1472c64688914149a318c7b8850a3df3 in term 0.
I20240502 07:58:57.172983 9645 raft_consensus.cc:2454] T d37b8a7f016647a7a03baedc37ed5419 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1472c64688914149a318c7b8850a3df3 in term 0.
I20240502 07:58:57.173236 9713 leader_election.cc:302] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1472c64688914149a318c7b8850a3df3, a201db721bda4dc3a4422dc8fa9bf72e; no voters:
I20240502 07:58:57.173400 10558 raft_consensus.cc:2781] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:57.173456 10558 raft_consensus.cc:482] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:57.173492 10558 raft_consensus.cc:3037] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:57.174224 10558 raft_consensus.cc:504] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:57.174520 10558 leader_election.cc:288] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 [CANDIDATE]: Term 1 election: Requested vote from peers a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551), 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667)
I20240502 07:58:57.174775 9514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d37b8a7f016647a7a03baedc37ed5419" candidate_uuid: "1472c64688914149a318c7b8850a3df3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a201db721bda4dc3a4422dc8fa9bf72e"
I20240502 07:58:57.174790 9645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d37b8a7f016647a7a03baedc37ed5419" candidate_uuid: "1472c64688914149a318c7b8850a3df3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8337af1943564c0dad18b45a597b805a"
I20240502 07:58:57.174866 9514 raft_consensus.cc:3037] T d37b8a7f016647a7a03baedc37ed5419 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:57.174882 9645 raft_consensus.cc:3037] T d37b8a7f016647a7a03baedc37ed5419 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:57.175637 9645 raft_consensus.cc:2454] T d37b8a7f016647a7a03baedc37ed5419 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1472c64688914149a318c7b8850a3df3 in term 1.
I20240502 07:58:57.175639 9514 raft_consensus.cc:2454] T d37b8a7f016647a7a03baedc37ed5419 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1472c64688914149a318c7b8850a3df3 in term 1.
I20240502 07:58:57.175904 9711 leader_election.cc:302] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1472c64688914149a318c7b8850a3df3, 8337af1943564c0dad18b45a597b805a; no voters:
I20240502 07:58:57.176055 10558 raft_consensus.cc:2781] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:57.176257 10558 raft_consensus.cc:686] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Becoming Leader. State: Replica: 1472c64688914149a318c7b8850a3df3, State: Running, Role: LEADER
I20240502 07:58:57.176383 10558 consensus_queue.cc:237] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:57.177779 9371 catalog_manager.cc:5484] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1472c64688914149a318c7b8850a3df3 (127.3.139.67). New cstate: current_term: 1 leader_uuid: "1472c64688914149a318c7b8850a3df3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:57.311663 10702 raft_consensus.cc:482] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:57.311791 10702 raft_consensus.cc:504] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:58:57.312212 10702 leader_election.cc:288] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191), 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379)
I20240502 07:58:57.312654 9776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a12a084dba23425996e9122c083c1444" candidate_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1472c64688914149a318c7b8850a3df3" is_pre_election: true
I20240502 07:58:57.312654 9907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a12a084dba23425996e9122c083c1444" candidate_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" is_pre_election: true
I20240502 07:58:57.312803 9907 raft_consensus.cc:2454] T a12a084dba23425996e9122c083c1444 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a201db721bda4dc3a4422dc8fa9bf72e in term 0.
I20240502 07:58:57.312803 9776 raft_consensus.cc:2454] T a12a084dba23425996e9122c083c1444 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a201db721bda4dc3a4422dc8fa9bf72e in term 0.
I20240502 07:58:57.313102 9449 leader_election.cc:302] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cc5b9962ed24441bcba0c07b9f5438e, a201db721bda4dc3a4422dc8fa9bf72e; no voters:
I20240502 07:58:57.313283 10702 raft_consensus.cc:2781] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:57.313354 10702 raft_consensus.cc:482] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:57.313391 10702 raft_consensus.cc:3037] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:57.314239 10702 raft_consensus.cc:504] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:58:57.314554 10702 leader_election.cc:288] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e [CANDIDATE]: Term 1 election: Requested vote from peers 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191), 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379)
I20240502 07:58:57.314862 9776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a12a084dba23425996e9122c083c1444" candidate_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1472c64688914149a318c7b8850a3df3"
I20240502 07:58:57.314860 9907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a12a084dba23425996e9122c083c1444" candidate_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cc5b9962ed24441bcba0c07b9f5438e"
I20240502 07:58:57.314980 9776 raft_consensus.cc:3037] T a12a084dba23425996e9122c083c1444 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:57.314996 9907 raft_consensus.cc:3037] T a12a084dba23425996e9122c083c1444 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:57.315886 9907 raft_consensus.cc:2454] T a12a084dba23425996e9122c083c1444 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a201db721bda4dc3a4422dc8fa9bf72e in term 1.
I20240502 07:58:57.315886 9776 raft_consensus.cc:2454] T a12a084dba23425996e9122c083c1444 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a201db721bda4dc3a4422dc8fa9bf72e in term 1.
I20240502 07:58:57.316164 9449 leader_election.cc:302] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cc5b9962ed24441bcba0c07b9f5438e, a201db721bda4dc3a4422dc8fa9bf72e; no voters:
I20240502 07:58:57.316319 10702 raft_consensus.cc:2781] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:57.316392 10702 raft_consensus.cc:686] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Becoming Leader. State: Replica: a201db721bda4dc3a4422dc8fa9bf72e, State: Running, Role: LEADER
I20240502 07:58:57.316532 10702 consensus_queue.cc:237] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:58:57.318145 9371 catalog_manager.cc:5484] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e reported cstate change: term changed from 0 to 1, leader changed from <none> to a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65). New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: HEALTHY } } }
I20240502 07:58:57.355089 10558 raft_consensus.cc:482] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:57.355211 10558 raft_consensus.cc:504] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:57.355551 10558 leader_election.cc:288] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667), 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379)
I20240502 07:58:57.355998 9645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96dab19eeaea4648913045482b3e5e9e" candidate_uuid: "1472c64688914149a318c7b8850a3df3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8337af1943564c0dad18b45a597b805a" is_pre_election: true
I20240502 07:58:57.355998 9907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96dab19eeaea4648913045482b3e5e9e" candidate_uuid: "1472c64688914149a318c7b8850a3df3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" is_pre_election: true
I20240502 07:58:57.356143 9645 raft_consensus.cc:2454] T 96dab19eeaea4648913045482b3e5e9e P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1472c64688914149a318c7b8850a3df3 in term 0.
I20240502 07:58:57.356143 9907 raft_consensus.cc:2454] T 96dab19eeaea4648913045482b3e5e9e P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1472c64688914149a318c7b8850a3df3 in term 0.
I20240502 07:58:57.356439 9711 leader_election.cc:302] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1472c64688914149a318c7b8850a3df3, 6cc5b9962ed24441bcba0c07b9f5438e; no voters:
I20240502 07:58:57.356616 10558 raft_consensus.cc:2781] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:57.356678 10558 raft_consensus.cc:482] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:57.356721 10558 raft_consensus.cc:3037] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:57.357445 10558 raft_consensus.cc:504] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:57.357738 10558 leader_election.cc:288] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3 [CANDIDATE]: Term 1 election: Requested vote from peers 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667), 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379)
I20240502 07:58:57.358072 9645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96dab19eeaea4648913045482b3e5e9e" candidate_uuid: "1472c64688914149a318c7b8850a3df3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8337af1943564c0dad18b45a597b805a"
I20240502 07:58:57.358115 9907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96dab19eeaea4648913045482b3e5e9e" candidate_uuid: "1472c64688914149a318c7b8850a3df3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cc5b9962ed24441bcba0c07b9f5438e"
I20240502 07:58:57.358204 9907 raft_consensus.cc:3037] T 96dab19eeaea4648913045482b3e5e9e P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:57.358201 9645 raft_consensus.cc:3037] T 96dab19eeaea4648913045482b3e5e9e P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:57.359082 9907 raft_consensus.cc:2454] T 96dab19eeaea4648913045482b3e5e9e P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1472c64688914149a318c7b8850a3df3 in term 1.
I20240502 07:58:57.359107 9645 raft_consensus.cc:2454] T 96dab19eeaea4648913045482b3e5e9e P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1472c64688914149a318c7b8850a3df3 in term 1.
I20240502 07:58:57.359360 9711 leader_election.cc:302] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1472c64688914149a318c7b8850a3df3, 6cc5b9962ed24441bcba0c07b9f5438e; no voters:
I20240502 07:58:57.359515 10558 raft_consensus.cc:2781] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:57.359598 10558 raft_consensus.cc:686] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Becoming Leader. State: Replica: 1472c64688914149a318c7b8850a3df3, State: Running, Role: LEADER
I20240502 07:58:57.359728 10558 consensus_queue.cc:237] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:57.361361 9371 catalog_manager.cc:5484] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1472c64688914149a318c7b8850a3df3 (127.3.139.67). New cstate: current_term: 1 leader_uuid: "1472c64688914149a318c7b8850a3df3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: UNKNOWN } } }
W20240502 07:58:57.477205 10732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:57.477424 10732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:57.480577 10732 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_3'
I20240502 07:58:57.549489 9776 raft_consensus.cc:1261] T a12a084dba23425996e9122c083c1444 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:57.550002 9907 raft_consensus.cc:1261] T a12a084dba23425996e9122c083c1444 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:57.550038 9908 raft_consensus.cc:1261] T 96dab19eeaea4648913045482b3e5e9e P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:57.550140 9645 raft_consensus.cc:1261] T 96dab19eeaea4648913045482b3e5e9e P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:57.550555 10558 consensus_queue.cc:1035] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:57.551209 10511 consensus_queue.cc:1035] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:57.551478 10511 consensus_queue.cc:1035] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:57.552238 10728 consensus_queue.cc:1035] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:57.552440 9646 raft_consensus.cc:1261] T d37b8a7f016647a7a03baedc37ed5419 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:57.556993 9514 raft_consensus.cc:1261] T d37b8a7f016647a7a03baedc37ed5419 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:57.557322 10558 consensus_queue.cc:1035] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:57.557519 10728 consensus_queue.cc:1035] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 87.8081 ms
time per row: 0.0439041 ms
I20240502 07:58:57.609334 9371 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47034:
name: "rebalance_test_table_4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:58:57.614470 9493 tablet_service.cc:1467] Processing CreateTablet for tablet ee2e79e677f643409e3c8dd49b9517cf (DEFAULT_TABLE table=rebalance_test_table_4 [id=a6211a0188464785886f1990ca644211]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:57.614552 9626 tablet_service.cc:1467] Processing CreateTablet for tablet ee2e79e677f643409e3c8dd49b9517cf (DEFAULT_TABLE table=rebalance_test_table_4 [id=a6211a0188464785886f1990ca644211]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:57.614769 9626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee2e79e677f643409e3c8dd49b9517cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.614770 9493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee2e79e677f643409e3c8dd49b9517cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.616766 10717 tablet_bootstrap.cc:492] T ee2e79e677f643409e3c8dd49b9517cf P 8337af1943564c0dad18b45a597b805a: Bootstrap starting.
I20240502 07:58:57.616868 10716 tablet_bootstrap.cc:492] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e: Bootstrap starting.
I20240502 07:58:57.617555 10716 tablet_bootstrap.cc:654] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:57.617555 10717 tablet_bootstrap.cc:654] T ee2e79e677f643409e3c8dd49b9517cf P 8337af1943564c0dad18b45a597b805a: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:57.618492 10717 tablet_bootstrap.cc:492] T ee2e79e677f643409e3c8dd49b9517cf P 8337af1943564c0dad18b45a597b805a: No bootstrap required, opened a new log
I20240502 07:58:57.618566 10717 ts_tablet_manager.cc:1367] T ee2e79e677f643409e3c8dd49b9517cf P 8337af1943564c0dad18b45a597b805a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:57.618808 9886 tablet_service.cc:1467] Processing CreateTablet for tablet ee2e79e677f643409e3c8dd49b9517cf (DEFAULT_TABLE table=rebalance_test_table_4 [id=a6211a0188464785886f1990ca644211]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:57.618950 10717 raft_consensus.cc:348] T ee2e79e677f643409e3c8dd49b9517cf P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:57.619037 10717 raft_consensus.cc:374] T ee2e79e677f643409e3c8dd49b9517cf P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:57.619081 9886 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee2e79e677f643409e3c8dd49b9517cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.619089 10717 raft_consensus.cc:729] T ee2e79e677f643409e3c8dd49b9517cf P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8337af1943564c0dad18b45a597b805a, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.619199 10717 consensus_queue.cc:260] T ee2e79e677f643409e3c8dd49b9517cf P 8337af1943564c0dad18b45a597b805a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:57.619426 10717 ts_tablet_manager.cc:1398] T ee2e79e677f643409e3c8dd49b9517cf P 8337af1943564c0dad18b45a597b805a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:57.620369 9887 tablet_service.cc:1467] Processing CreateTablet for tablet 7da9d3b7fe8c40f6a68bb1b7a926d994 (DEFAULT_TABLE table=rebalance_test_table_4 [id=a6211a0188464785886f1990ca644211]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:57.620574 9887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7da9d3b7fe8c40f6a68bb1b7a926d994. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.620654 10721 tablet_bootstrap.cc:492] T ee2e79e677f643409e3c8dd49b9517cf P 6cc5b9962ed24441bcba0c07b9f5438e: Bootstrap starting.
I20240502 07:58:57.621279 10721 tablet_bootstrap.cc:654] T ee2e79e677f643409e3c8dd49b9517cf P 6cc5b9962ed24441bcba0c07b9f5438e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:57.622076 10721 tablet_bootstrap.cc:492] T ee2e79e677f643409e3c8dd49b9517cf P 6cc5b9962ed24441bcba0c07b9f5438e: No bootstrap required, opened a new log
I20240502 07:58:57.622150 10721 ts_tablet_manager.cc:1367] T ee2e79e677f643409e3c8dd49b9517cf P 6cc5b9962ed24441bcba0c07b9f5438e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:57.622320 9626 tablet_service.cc:1467] Processing CreateTablet for tablet 140868fcb19a4e7c837ce0c2c1c7041e (DEFAULT_TABLE table=rebalance_test_table_4 [id=a6211a0188464785886f1990ca644211]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:57.622515 10721 raft_consensus.cc:348] T ee2e79e677f643409e3c8dd49b9517cf P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:57.622561 9626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 140868fcb19a4e7c837ce0c2c1c7041e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.622598 10721 raft_consensus.cc:374] T ee2e79e677f643409e3c8dd49b9517cf P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:57.622633 10721 raft_consensus.cc:729] T ee2e79e677f643409e3c8dd49b9517cf P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cc5b9962ed24441bcba0c07b9f5438e, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.622743 10721 consensus_queue.cc:260] T ee2e79e677f643409e3c8dd49b9517cf P 6cc5b9962ed24441bcba0c07b9f5438e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:57.622985 10721 ts_tablet_manager.cc:1398] T ee2e79e677f643409e3c8dd49b9517cf P 6cc5b9962ed24441bcba0c07b9f5438e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:57.623145 10721 tablet_bootstrap.cc:492] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e: Bootstrap starting.
I20240502 07:58:57.623733 10721 tablet_bootstrap.cc:654] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:57.624028 10717 tablet_bootstrap.cc:492] T 140868fcb19a4e7c837ce0c2c1c7041e P 8337af1943564c0dad18b45a597b805a: Bootstrap starting.
I20240502 07:58:57.624661 9756 tablet_service.cc:1467] Processing CreateTablet for tablet 140868fcb19a4e7c837ce0c2c1c7041e (DEFAULT_TABLE table=rebalance_test_table_4 [id=a6211a0188464785886f1990ca644211]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:57.624796 10717 tablet_bootstrap.cc:654] T 140868fcb19a4e7c837ce0c2c1c7041e P 8337af1943564c0dad18b45a597b805a: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:57.624897 9756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 140868fcb19a4e7c837ce0c2c1c7041e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.624933 9495 tablet_service.cc:1467] Processing CreateTablet for tablet 7da9d3b7fe8c40f6a68bb1b7a926d994 (DEFAULT_TABLE table=rebalance_test_table_4 [id=a6211a0188464785886f1990ca644211]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:57.624969 9757 tablet_service.cc:1467] Processing CreateTablet for tablet 7da9d3b7fe8c40f6a68bb1b7a926d994 (DEFAULT_TABLE table=rebalance_test_table_4 [id=a6211a0188464785886f1990ca644211]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:57.625128 9495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7da9d3b7fe8c40f6a68bb1b7a926d994. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.625208 9757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7da9d3b7fe8c40f6a68bb1b7a926d994. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.626586 10715 tablet_bootstrap.cc:492] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 1472c64688914149a318c7b8850a3df3: Bootstrap starting.
I20240502 07:58:57.627364 10715 tablet_bootstrap.cc:654] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 1472c64688914149a318c7b8850a3df3: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:57.627506 10716 tablet_bootstrap.cc:492] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e: No bootstrap required, opened a new log
I20240502 07:58:57.627591 10716 ts_tablet_manager.cc:1367] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20240502 07:58:57.628002 10716 raft_consensus.cc:348] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:57.628105 10716 raft_consensus.cc:374] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:57.628136 10716 raft_consensus.cc:729] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a201db721bda4dc3a4422dc8fa9bf72e, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.628252 10716 consensus_queue.cc:260] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:57.628341 10715 tablet_bootstrap.cc:492] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 1472c64688914149a318c7b8850a3df3: No bootstrap required, opened a new log
I20240502 07:58:57.628407 10715 ts_tablet_manager.cc:1367] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 1472c64688914149a318c7b8850a3df3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:57.628521 10716 ts_tablet_manager.cc:1398] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:57.628772 10716 tablet_bootstrap.cc:492] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P a201db721bda4dc3a4422dc8fa9bf72e: Bootstrap starting.
I20240502 07:58:57.628782 10715 raft_consensus.cc:348] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:57.628809 9493 tablet_service.cc:1467] Processing CreateTablet for tablet 140868fcb19a4e7c837ce0c2c1c7041e (DEFAULT_TABLE table=rebalance_test_table_4 [id=a6211a0188464785886f1990ca644211]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:57.628883 10715 raft_consensus.cc:374] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:57.628911 10715 raft_consensus.cc:729] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1472c64688914149a318c7b8850a3df3, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.629010 9493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 140868fcb19a4e7c837ce0c2c1c7041e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.629019 10715 consensus_queue.cc:260] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 1472c64688914149a318c7b8850a3df3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:57.629257 10715 ts_tablet_manager.cc:1398] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 1472c64688914149a318c7b8850a3df3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:57.629405 10715 tablet_bootstrap.cc:492] T 140868fcb19a4e7c837ce0c2c1c7041e P 1472c64688914149a318c7b8850a3df3: Bootstrap starting.
I20240502 07:58:57.629573 10716 tablet_bootstrap.cc:654] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P a201db721bda4dc3a4422dc8fa9bf72e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:57.630146 10715 tablet_bootstrap.cc:654] T 140868fcb19a4e7c837ce0c2c1c7041e P 1472c64688914149a318c7b8850a3df3: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:57.631423 10717 tablet_bootstrap.cc:492] T 140868fcb19a4e7c837ce0c2c1c7041e P 8337af1943564c0dad18b45a597b805a: No bootstrap required, opened a new log
I20240502 07:58:57.631505 10717 ts_tablet_manager.cc:1367] T 140868fcb19a4e7c837ce0c2c1c7041e P 8337af1943564c0dad18b45a597b805a: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240502 07:58:57.632198 10721 tablet_bootstrap.cc:492] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e: No bootstrap required, opened a new log
I20240502 07:58:57.632200 10717 raft_consensus.cc:348] T 140868fcb19a4e7c837ce0c2c1c7041e P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:57.632278 10721 ts_tablet_manager.cc:1367] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20240502 07:58:57.632294 10717 raft_consensus.cc:374] T 140868fcb19a4e7c837ce0c2c1c7041e P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:57.632318 10717 raft_consensus.cc:729] T 140868fcb19a4e7c837ce0c2c1c7041e P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8337af1943564c0dad18b45a597b805a, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.632427 10717 consensus_queue.cc:260] T 140868fcb19a4e7c837ce0c2c1c7041e P 8337af1943564c0dad18b45a597b805a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:57.632656 10717 ts_tablet_manager.cc:1398] T 140868fcb19a4e7c837ce0c2c1c7041e P 8337af1943564c0dad18b45a597b805a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:57.632665 10721 raft_consensus.cc:348] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:57.632756 10721 raft_consensus.cc:374] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:57.632784 10721 raft_consensus.cc:729] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cc5b9962ed24441bcba0c07b9f5438e, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.632905 10721 consensus_queue.cc:260] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:57.633147 10721 ts_tablet_manager.cc:1398] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:57.633549 10716 tablet_bootstrap.cc:492] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P a201db721bda4dc3a4422dc8fa9bf72e: No bootstrap required, opened a new log
I20240502 07:58:57.633623 10716 ts_tablet_manager.cc:1367] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P a201db721bda4dc3a4422dc8fa9bf72e: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 07:58:57.633980 10716 raft_consensus.cc:348] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:57.634047 10716 raft_consensus.cc:374] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:57.634065 10716 raft_consensus.cc:729] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a201db721bda4dc3a4422dc8fa9bf72e, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.634155 10716 consensus_queue.cc:260] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P a201db721bda4dc3a4422dc8fa9bf72e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:57.634393 10716 ts_tablet_manager.cc:1398] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P a201db721bda4dc3a4422dc8fa9bf72e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:57.634565 10716 tablet_bootstrap.cc:492] T 140868fcb19a4e7c837ce0c2c1c7041e P a201db721bda4dc3a4422dc8fa9bf72e: Bootstrap starting.
I20240502 07:58:57.635260 10715 tablet_bootstrap.cc:492] T 140868fcb19a4e7c837ce0c2c1c7041e P 1472c64688914149a318c7b8850a3df3: No bootstrap required, opened a new log
I20240502 07:58:57.635288 10716 tablet_bootstrap.cc:654] T 140868fcb19a4e7c837ce0c2c1c7041e P a201db721bda4dc3a4422dc8fa9bf72e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:57.635674 10715 ts_tablet_manager.cc:1367] T 140868fcb19a4e7c837ce0c2c1c7041e P 1472c64688914149a318c7b8850a3df3: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 07:58:57.636118 10715 raft_consensus.cc:348] T 140868fcb19a4e7c837ce0c2c1c7041e P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:57.636265 10715 raft_consensus.cc:374] T 140868fcb19a4e7c837ce0c2c1c7041e P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:57.636355 10715 raft_consensus.cc:729] T 140868fcb19a4e7c837ce0c2c1c7041e P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1472c64688914149a318c7b8850a3df3, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.636512 10715 consensus_queue.cc:260] T 140868fcb19a4e7c837ce0c2c1c7041e P 1472c64688914149a318c7b8850a3df3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:57.636621 10716 tablet_bootstrap.cc:492] T 140868fcb19a4e7c837ce0c2c1c7041e P a201db721bda4dc3a4422dc8fa9bf72e: No bootstrap required, opened a new log
I20240502 07:58:57.636698 10716 ts_tablet_manager.cc:1367] T 140868fcb19a4e7c837ce0c2c1c7041e P a201db721bda4dc3a4422dc8fa9bf72e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:57.636754 10715 ts_tablet_manager.cc:1398] T 140868fcb19a4e7c837ce0c2c1c7041e P 1472c64688914149a318c7b8850a3df3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:57.637071 10716 raft_consensus.cc:348] T 140868fcb19a4e7c837ce0c2c1c7041e P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:57.637190 10716 raft_consensus.cc:374] T 140868fcb19a4e7c837ce0c2c1c7041e P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:57.637235 10716 raft_consensus.cc:729] T 140868fcb19a4e7c837ce0c2c1c7041e P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a201db721bda4dc3a4422dc8fa9bf72e, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.637372 10716 consensus_queue.cc:260] T 140868fcb19a4e7c837ce0c2c1c7041e P a201db721bda4dc3a4422dc8fa9bf72e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:57.637625 10716 ts_tablet_manager.cc:1398] T 140868fcb19a4e7c837ce0c2c1c7041e P a201db721bda4dc3a4422dc8fa9bf72e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:57.682901 10702 raft_consensus.cc:482] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:57.683018 10702 raft_consensus.cc:504] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:57.683395 10702 leader_election.cc:288] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667), 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379)
I20240502 07:58:57.683820 9908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee2e79e677f643409e3c8dd49b9517cf" candidate_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" is_pre_election: true
I20240502 07:58:57.683811 9646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee2e79e677f643409e3c8dd49b9517cf" candidate_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8337af1943564c0dad18b45a597b805a" is_pre_election: true
I20240502 07:58:57.683962 9908 raft_consensus.cc:2454] T ee2e79e677f643409e3c8dd49b9517cf P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a201db721bda4dc3a4422dc8fa9bf72e in term 0.
I20240502 07:58:57.683962 9646 raft_consensus.cc:2454] T ee2e79e677f643409e3c8dd49b9517cf P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a201db721bda4dc3a4422dc8fa9bf72e in term 0.
I20240502 07:58:57.684206 9449 leader_election.cc:302] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8337af1943564c0dad18b45a597b805a, a201db721bda4dc3a4422dc8fa9bf72e; no voters:
I20240502 07:58:57.684366 10702 raft_consensus.cc:2781] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:57.684425 10702 raft_consensus.cc:482] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:57.684471 10702 raft_consensus.cc:3037] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:57.685240 10702 raft_consensus.cc:504] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:57.685552 10702 leader_election.cc:288] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [CANDIDATE]: Term 1 election: Requested vote from peers 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667), 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379)
I20240502 07:58:57.685837 9646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee2e79e677f643409e3c8dd49b9517cf" candidate_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8337af1943564c0dad18b45a597b805a"
I20240502 07:58:57.685958 9646 raft_consensus.cc:3037] T ee2e79e677f643409e3c8dd49b9517cf P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:57.686020 9908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee2e79e677f643409e3c8dd49b9517cf" candidate_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cc5b9962ed24441bcba0c07b9f5438e"
I20240502 07:58:57.686127 9908 raft_consensus.cc:3037] T ee2e79e677f643409e3c8dd49b9517cf P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:57.686816 9646 raft_consensus.cc:2454] T ee2e79e677f643409e3c8dd49b9517cf P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a201db721bda4dc3a4422dc8fa9bf72e in term 1.
I20240502 07:58:57.686906 9908 raft_consensus.cc:2454] T ee2e79e677f643409e3c8dd49b9517cf P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a201db721bda4dc3a4422dc8fa9bf72e in term 1.
I20240502 07:58:57.687125 9449 leader_election.cc:302] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8337af1943564c0dad18b45a597b805a, a201db721bda4dc3a4422dc8fa9bf72e; no voters:
I20240502 07:58:57.687276 10702 raft_consensus.cc:2781] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:57.687369 10702 raft_consensus.cc:686] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Becoming Leader. State: Replica: a201db721bda4dc3a4422dc8fa9bf72e, State: Running, Role: LEADER
I20240502 07:58:57.687503 10702 consensus_queue.cc:237] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:57.688944 9372 catalog_manager.cc:5484] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e reported cstate change: term changed from 0 to 1, leader changed from <none> to a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65). New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:57.944473 10702 raft_consensus.cc:482] T 140868fcb19a4e7c837ce0c2c1c7041e P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:57.944591 10702 raft_consensus.cc:504] T 140868fcb19a4e7c837ce0c2c1c7041e P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:57.945008 10702 leader_election.cc:288] T 140868fcb19a4e7c837ce0c2c1c7041e P a201db721bda4dc3a4422dc8fa9bf72e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191), 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667)
I20240502 07:58:57.945422 9646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "140868fcb19a4e7c837ce0c2c1c7041e" candidate_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8337af1943564c0dad18b45a597b805a" is_pre_election: true
I20240502 07:58:57.945415 9776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "140868fcb19a4e7c837ce0c2c1c7041e" candidate_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1472c64688914149a318c7b8850a3df3" is_pre_election: true
I20240502 07:58:57.945551 9646 raft_consensus.cc:2454] T 140868fcb19a4e7c837ce0c2c1c7041e P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a201db721bda4dc3a4422dc8fa9bf72e in term 0.
I20240502 07:58:57.945577 9776 raft_consensus.cc:2454] T 140868fcb19a4e7c837ce0c2c1c7041e P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a201db721bda4dc3a4422dc8fa9bf72e in term 0.
I20240502 07:58:57.945842 9451 leader_election.cc:302] T 140868fcb19a4e7c837ce0c2c1c7041e P a201db721bda4dc3a4422dc8fa9bf72e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1472c64688914149a318c7b8850a3df3, a201db721bda4dc3a4422dc8fa9bf72e; no voters:
I20240502 07:58:57.946041 10702 raft_consensus.cc:2781] T 140868fcb19a4e7c837ce0c2c1c7041e P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:57.946111 10702 raft_consensus.cc:482] T 140868fcb19a4e7c837ce0c2c1c7041e P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:57.946146 10702 raft_consensus.cc:3037] T 140868fcb19a4e7c837ce0c2c1c7041e P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:57.946884 10702 raft_consensus.cc:504] T 140868fcb19a4e7c837ce0c2c1c7041e P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:57.947201 10702 leader_election.cc:288] T 140868fcb19a4e7c837ce0c2c1c7041e P a201db721bda4dc3a4422dc8fa9bf72e [CANDIDATE]: Term 1 election: Requested vote from peers 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191), 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667)
I20240502 07:58:57.947466 9776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "140868fcb19a4e7c837ce0c2c1c7041e" candidate_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1472c64688914149a318c7b8850a3df3"
I20240502 07:58:57.947575 9776 raft_consensus.cc:3037] T 140868fcb19a4e7c837ce0c2c1c7041e P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:57.947633 9646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "140868fcb19a4e7c837ce0c2c1c7041e" candidate_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8337af1943564c0dad18b45a597b805a"
I20240502 07:58:57.947738 9646 raft_consensus.cc:3037] T 140868fcb19a4e7c837ce0c2c1c7041e P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:57.948339 9776 raft_consensus.cc:2454] T 140868fcb19a4e7c837ce0c2c1c7041e P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a201db721bda4dc3a4422dc8fa9bf72e in term 1.
I20240502 07:58:57.948563 9646 raft_consensus.cc:2454] T 140868fcb19a4e7c837ce0c2c1c7041e P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a201db721bda4dc3a4422dc8fa9bf72e in term 1.
I20240502 07:58:57.948606 9451 leader_election.cc:302] T 140868fcb19a4e7c837ce0c2c1c7041e P a201db721bda4dc3a4422dc8fa9bf72e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1472c64688914149a318c7b8850a3df3, a201db721bda4dc3a4422dc8fa9bf72e; no voters:
I20240502 07:58:57.948748 10702 raft_consensus.cc:2781] T 140868fcb19a4e7c837ce0c2c1c7041e P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:57.948834 10702 raft_consensus.cc:686] T 140868fcb19a4e7c837ce0c2c1c7041e P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Becoming Leader. State: Replica: a201db721bda4dc3a4422dc8fa9bf72e, State: Running, Role: LEADER
I20240502 07:58:57.948968 10702 consensus_queue.cc:237] T 140868fcb19a4e7c837ce0c2c1c7041e P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:57.950598 9372 catalog_manager.cc:5484] T 140868fcb19a4e7c837ce0c2c1c7041e P a201db721bda4dc3a4422dc8fa9bf72e reported cstate change: term changed from 0 to 1, leader changed from <none> to a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65). New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:57.986817 10514 raft_consensus.cc:482] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:57.986950 10514 raft_consensus.cc:504] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:57.987448 10514 leader_election.cc:288] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551), 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191)
I20240502 07:58:57.987968 9514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7da9d3b7fe8c40f6a68bb1b7a926d994" candidate_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" is_pre_election: true
I20240502 07:58:57.988121 9514 raft_consensus.cc:2454] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cc5b9962ed24441bcba0c07b9f5438e in term 0.
I20240502 07:58:57.988405 9844 leader_election.cc:302] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cc5b9962ed24441bcba0c07b9f5438e, a201db721bda4dc3a4422dc8fa9bf72e; no voters:
I20240502 07:58:57.988592 10514 raft_consensus.cc:2781] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:57.988646 10514 raft_consensus.cc:482] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:57.988687 10514 raft_consensus.cc:3037] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:57.989543 10514 raft_consensus.cc:504] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:57.989869 10514 leader_election.cc:288] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e [CANDIDATE]: Term 1 election: Requested vote from peers a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551), 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191)
I20240502 07:58:57.990187 9514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7da9d3b7fe8c40f6a68bb1b7a926d994" candidate_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a201db721bda4dc3a4422dc8fa9bf72e"
I20240502 07:58:57.990300 9514 raft_consensus.cc:3037] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:57.991214 9514 raft_consensus.cc:2454] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cc5b9962ed24441bcba0c07b9f5438e in term 1.
I20240502 07:58:57.991278 9776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7da9d3b7fe8c40f6a68bb1b7a926d994" candidate_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1472c64688914149a318c7b8850a3df3" is_pre_election: true
I20240502 07:58:57.991350 9777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7da9d3b7fe8c40f6a68bb1b7a926d994" candidate_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1472c64688914149a318c7b8850a3df3"
I20240502 07:58:57.991436 9776 raft_consensus.cc:2454] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cc5b9962ed24441bcba0c07b9f5438e in term 0.
I20240502 07:58:57.991513 9844 leader_election.cc:302] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cc5b9962ed24441bcba0c07b9f5438e, a201db721bda4dc3a4422dc8fa9bf72e; no voters:
I20240502 07:58:57.991657 10514 raft_consensus.cc:2781] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:57.991814 10514 raft_consensus.cc:686] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 LEADER]: Becoming Leader. State: Replica: 6cc5b9962ed24441bcba0c07b9f5438e, State: Running, Role: LEADER
I20240502 07:58:57.991956 10514 consensus_queue.cc:237] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:57.993484 9372 catalog_manager.cc:5484] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68). New cstate: current_term: 1 leader_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:58.100615 10702 consensus_queue.cc:1035] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:58.104535 10511 consensus_queue.cc:1035] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 07:58:58.190713 10784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:58.190924 10784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:58.193822 10784 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_4'
I20240502 07:58:58.263654 9777 raft_consensus.cc:1261] T 140868fcb19a4e7c837ce0c2c1c7041e P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:58.264430 10702 consensus_queue.cc:1035] T 140868fcb19a4e7c837ce0c2c1c7041e P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:58.265419 9776 raft_consensus.cc:3037] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:58.265515 9646 raft_consensus.cc:1261] T 140868fcb19a4e7c837ce0c2c1c7041e P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:58.266175 9514 raft_consensus.cc:1261] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:58.266475 9776 raft_consensus.cc:1261] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:58.268536 10702 consensus_queue.cc:1035] T 140868fcb19a4e7c837ce0c2c1c7041e P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:58.270021 10782 consensus_queue.cc:1035] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:58.273593 10782 consensus_queue.cc:1035] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:58.299389 10786 mvcc.cc:205] Tried to move back new op lower bound from 7023152079923085312 to 7023152078640291840. Current Snapshot: MvccSnapshot[applied={T|T < 7023152079923085312 or (T in {7023152079923085312})}]
I20240502 07:58:58.310652 10821 mvcc.cc:205] Tried to move back new op lower bound from 7023152079950798848 to 7023152078816374784. Current Snapshot: MvccSnapshot[applied={T|T < 7023152079931281408}]
INSERT report
rows total: 2000
time total: 95.2739 ms
time per row: 0.047637 ms
I20240502 07:58:58.326726 9373 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47034:
name: "rebalance_test_table_5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:58:58.331864 9756 tablet_service.cc:1467] Processing CreateTablet for tablet ecd0995c31fe4e708712c62ca60511c9 (DEFAULT_TABLE table=rebalance_test_table_5 [id=51b1431ec18d4bb8b4d5ae1909ae1322]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:58.331913 9626 tablet_service.cc:1467] Processing CreateTablet for tablet ecd0995c31fe4e708712c62ca60511c9 (DEFAULT_TABLE table=rebalance_test_table_5 [id=51b1431ec18d4bb8b4d5ae1909ae1322]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:58.332166 9756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecd0995c31fe4e708712c62ca60511c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:58.332166 9626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecd0995c31fe4e708712c62ca60511c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:58.332370 9757 tablet_service.cc:1467] Processing CreateTablet for tablet d3db04c54a184eb8a1c23fec9e7145eb (DEFAULT_TABLE table=rebalance_test_table_5 [id=51b1431ec18d4bb8b4d5ae1909ae1322]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:58.332568 9757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3db04c54a184eb8a1c23fec9e7145eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:58.334124 10842 tablet_bootstrap.cc:492] T d3db04c54a184eb8a1c23fec9e7145eb P 1472c64688914149a318c7b8850a3df3: Bootstrap starting.
I20240502 07:58:58.334627 9887 tablet_service.cc:1467] Processing CreateTablet for tablet ecd0995c31fe4e708712c62ca60511c9 (DEFAULT_TABLE table=rebalance_test_table_5 [id=51b1431ec18d4bb8b4d5ae1909ae1322]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:58.334899 9887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecd0995c31fe4e708712c62ca60511c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:58.334982 10842 tablet_bootstrap.cc:654] T d3db04c54a184eb8a1c23fec9e7145eb P 1472c64688914149a318c7b8850a3df3: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:58.334935 9493 tablet_service.cc:1467] Processing CreateTablet for tablet 12e3bfe638d249c0934feda41b8a8876 (DEFAULT_TABLE table=rebalance_test_table_5 [id=51b1431ec18d4bb8b4d5ae1909ae1322]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:58.335157 9625 tablet_service.cc:1467] Processing CreateTablet for tablet 12e3bfe638d249c0934feda41b8a8876 (DEFAULT_TABLE table=rebalance_test_table_5 [id=51b1431ec18d4bb8b4d5ae1909ae1322]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:58.335244 9493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12e3bfe638d249c0934feda41b8a8876. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:58.335366 9625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12e3bfe638d249c0934feda41b8a8876. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:58.336102 10842 tablet_bootstrap.cc:492] T d3db04c54a184eb8a1c23fec9e7145eb P 1472c64688914149a318c7b8850a3df3: No bootstrap required, opened a new log
I20240502 07:58:58.336177 10842 ts_tablet_manager.cc:1367] T d3db04c54a184eb8a1c23fec9e7145eb P 1472c64688914149a318c7b8850a3df3: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 07:58:58.336616 10842 raft_consensus.cc:348] T d3db04c54a184eb8a1c23fec9e7145eb P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:58.336712 10842 raft_consensus.cc:374] T d3db04c54a184eb8a1c23fec9e7145eb P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:58.336740 10842 raft_consensus.cc:729] T d3db04c54a184eb8a1c23fec9e7145eb P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1472c64688914149a318c7b8850a3df3, State: Initialized, Role: FOLLOWER
I20240502 07:58:58.336865 10842 consensus_queue.cc:260] T d3db04c54a184eb8a1c23fec9e7145eb P 1472c64688914149a318c7b8850a3df3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:58.337128 10842 ts_tablet_manager.cc:1398] T d3db04c54a184eb8a1c23fec9e7145eb P 1472c64688914149a318c7b8850a3df3: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:58.337226 9886 tablet_service.cc:1467] Processing CreateTablet for tablet d3db04c54a184eb8a1c23fec9e7145eb (DEFAULT_TABLE table=rebalance_test_table_5 [id=51b1431ec18d4bb8b4d5ae1909ae1322]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:58.337284 10842 tablet_bootstrap.cc:492] T ecd0995c31fe4e708712c62ca60511c9 P 1472c64688914149a318c7b8850a3df3: Bootstrap starting.
I20240502 07:58:58.337460 9886 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3db04c54a184eb8a1c23fec9e7145eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:58.337671 9626 tablet_service.cc:1467] Processing CreateTablet for tablet d3db04c54a184eb8a1c23fec9e7145eb (DEFAULT_TABLE table=rebalance_test_table_5 [id=51b1431ec18d4bb8b4d5ae1909ae1322]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:58.337874 9626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3db04c54a184eb8a1c23fec9e7145eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:58.337990 10842 tablet_bootstrap.cc:654] T ecd0995c31fe4e708712c62ca60511c9 P 1472c64688914149a318c7b8850a3df3: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:58.339402 9888 tablet_service.cc:1467] Processing CreateTablet for tablet 12e3bfe638d249c0934feda41b8a8876 (DEFAULT_TABLE table=rebalance_test_table_5 [id=51b1431ec18d4bb8b4d5ae1909ae1322]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:58.339612 9888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12e3bfe638d249c0934feda41b8a8876. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:58.339671 10842 tablet_bootstrap.cc:492] T ecd0995c31fe4e708712c62ca60511c9 P 1472c64688914149a318c7b8850a3df3: No bootstrap required, opened a new log
I20240502 07:58:58.339699 10845 tablet_bootstrap.cc:492] T ecd0995c31fe4e708712c62ca60511c9 P 6cc5b9962ed24441bcba0c07b9f5438e: Bootstrap starting.
I20240502 07:58:58.339741 10842 ts_tablet_manager.cc:1367] T ecd0995c31fe4e708712c62ca60511c9 P 1472c64688914149a318c7b8850a3df3: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 07:58:58.340140 10842 raft_consensus.cc:348] T ecd0995c31fe4e708712c62ca60511c9 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:58.340230 10842 raft_consensus.cc:374] T ecd0995c31fe4e708712c62ca60511c9 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:58.340258 10842 raft_consensus.cc:729] T ecd0995c31fe4e708712c62ca60511c9 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1472c64688914149a318c7b8850a3df3, State: Initialized, Role: FOLLOWER
I20240502 07:58:58.340551 10845 tablet_bootstrap.cc:654] T ecd0995c31fe4e708712c62ca60511c9 P 6cc5b9962ed24441bcba0c07b9f5438e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:58.341187 10843 tablet_bootstrap.cc:492] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a: Bootstrap starting.
I20240502 07:58:58.341969 10843 tablet_bootstrap.cc:654] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:58.342226 10846 tablet_bootstrap.cc:492] T 12e3bfe638d249c0934feda41b8a8876 P a201db721bda4dc3a4422dc8fa9bf72e: Bootstrap starting.
I20240502 07:58:58.342411 10845 tablet_bootstrap.cc:492] T ecd0995c31fe4e708712c62ca60511c9 P 6cc5b9962ed24441bcba0c07b9f5438e: No bootstrap required, opened a new log
I20240502 07:58:58.343159 10846 tablet_bootstrap.cc:654] T 12e3bfe638d249c0934feda41b8a8876 P a201db721bda4dc3a4422dc8fa9bf72e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:58.343297 10845 ts_tablet_manager.cc:1367] T ecd0995c31fe4e708712c62ca60511c9 P 6cc5b9962ed24441bcba0c07b9f5438e: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 07:58:58.343299 10842 consensus_queue.cc:260] T ecd0995c31fe4e708712c62ca60511c9 P 1472c64688914149a318c7b8850a3df3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:58.343843 10845 raft_consensus.cc:348] T ecd0995c31fe4e708712c62ca60511c9 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:58.344077 10845 raft_consensus.cc:374] T ecd0995c31fe4e708712c62ca60511c9 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:58.344228 10846 tablet_bootstrap.cc:492] T 12e3bfe638d249c0934feda41b8a8876 P a201db721bda4dc3a4422dc8fa9bf72e: No bootstrap required, opened a new log
I20240502 07:58:58.344303 10846 ts_tablet_manager.cc:1367] T 12e3bfe638d249c0934feda41b8a8876 P a201db721bda4dc3a4422dc8fa9bf72e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:58.344734 10846 raft_consensus.cc:348] T 12e3bfe638d249c0934feda41b8a8876 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:58.344833 10846 raft_consensus.cc:374] T 12e3bfe638d249c0934feda41b8a8876 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:58.344864 10846 raft_consensus.cc:729] T 12e3bfe638d249c0934feda41b8a8876 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a201db721bda4dc3a4422dc8fa9bf72e, State: Initialized, Role: FOLLOWER
I20240502 07:58:58.344158 10845 raft_consensus.cc:729] T ecd0995c31fe4e708712c62ca60511c9 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cc5b9962ed24441bcba0c07b9f5438e, State: Initialized, Role: FOLLOWER
I20240502 07:58:58.344975 10846 consensus_queue.cc:260] T 12e3bfe638d249c0934feda41b8a8876 P a201db721bda4dc3a4422dc8fa9bf72e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:58.345109 10845 consensus_queue.cc:260] T ecd0995c31fe4e708712c62ca60511c9 P 6cc5b9962ed24441bcba0c07b9f5438e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:58.345249 10846 ts_tablet_manager.cc:1398] T 12e3bfe638d249c0934feda41b8a8876 P a201db721bda4dc3a4422dc8fa9bf72e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:58.345536 10845 ts_tablet_manager.cc:1398] T ecd0995c31fe4e708712c62ca60511c9 P 6cc5b9962ed24441bcba0c07b9f5438e: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:58.345721 10845 tablet_bootstrap.cc:492] T d3db04c54a184eb8a1c23fec9e7145eb P 6cc5b9962ed24441bcba0c07b9f5438e: Bootstrap starting.
I20240502 07:58:58.345932 10842 ts_tablet_manager.cc:1398] T ecd0995c31fe4e708712c62ca60511c9 P 1472c64688914149a318c7b8850a3df3: Time spent starting tablet: real 0.006s user 0.000s sys 0.001s
I20240502 07:58:58.346864 10845 tablet_bootstrap.cc:654] T d3db04c54a184eb8a1c23fec9e7145eb P 6cc5b9962ed24441bcba0c07b9f5438e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:58.348691 10843 tablet_bootstrap.cc:492] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a: No bootstrap required, opened a new log
I20240502 07:58:58.348825 10843 ts_tablet_manager.cc:1367] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 07:58:58.349264 10843 raft_consensus.cc:348] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:58.349373 10843 raft_consensus.cc:374] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:58.349403 10843 raft_consensus.cc:729] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8337af1943564c0dad18b45a597b805a, State: Initialized, Role: FOLLOWER
I20240502 07:58:58.349520 10843 consensus_queue.cc:260] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:58.349748 10843 ts_tablet_manager.cc:1398] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:58.349918 10843 tablet_bootstrap.cc:492] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a: Bootstrap starting.
I20240502 07:58:58.350916 10843 tablet_bootstrap.cc:654] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:58.352155 10843 tablet_bootstrap.cc:492] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a: No bootstrap required, opened a new log
I20240502 07:58:58.352250 10843 ts_tablet_manager.cc:1367] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:58.352656 10843 raft_consensus.cc:348] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:58.352757 10843 raft_consensus.cc:374] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:58.352797 10843 raft_consensus.cc:729] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8337af1943564c0dad18b45a597b805a, State: Initialized, Role: FOLLOWER
I20240502 07:58:58.352916 10843 consensus_queue.cc:260] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:58.353164 10843 ts_tablet_manager.cc:1398] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:58.353376 10843 tablet_bootstrap.cc:492] T d3db04c54a184eb8a1c23fec9e7145eb P 8337af1943564c0dad18b45a597b805a: Bootstrap starting.
I20240502 07:58:58.353621 10845 tablet_bootstrap.cc:492] T d3db04c54a184eb8a1c23fec9e7145eb P 6cc5b9962ed24441bcba0c07b9f5438e: No bootstrap required, opened a new log
I20240502 07:58:58.353704 10845 ts_tablet_manager.cc:1367] T d3db04c54a184eb8a1c23fec9e7145eb P 6cc5b9962ed24441bcba0c07b9f5438e: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 07:58:58.354189 10843 tablet_bootstrap.cc:654] T d3db04c54a184eb8a1c23fec9e7145eb P 8337af1943564c0dad18b45a597b805a: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:58.354158 10845 raft_consensus.cc:348] T d3db04c54a184eb8a1c23fec9e7145eb P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:58.354254 10845 raft_consensus.cc:374] T d3db04c54a184eb8a1c23fec9e7145eb P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:58.354316 10845 raft_consensus.cc:729] T d3db04c54a184eb8a1c23fec9e7145eb P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cc5b9962ed24441bcba0c07b9f5438e, State: Initialized, Role: FOLLOWER
I20240502 07:58:58.354466 10845 consensus_queue.cc:260] T d3db04c54a184eb8a1c23fec9e7145eb P 6cc5b9962ed24441bcba0c07b9f5438e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:58.354841 10845 ts_tablet_manager.cc:1398] T d3db04c54a184eb8a1c23fec9e7145eb P 6cc5b9962ed24441bcba0c07b9f5438e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:58.355084 10845 tablet_bootstrap.cc:492] T 12e3bfe638d249c0934feda41b8a8876 P 6cc5b9962ed24441bcba0c07b9f5438e: Bootstrap starting.
I20240502 07:58:58.355902 10845 tablet_bootstrap.cc:654] T 12e3bfe638d249c0934feda41b8a8876 P 6cc5b9962ed24441bcba0c07b9f5438e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:58.356254 10843 tablet_bootstrap.cc:492] T d3db04c54a184eb8a1c23fec9e7145eb P 8337af1943564c0dad18b45a597b805a: No bootstrap required, opened a new log
I20240502 07:58:58.356346 10843 ts_tablet_manager.cc:1367] T d3db04c54a184eb8a1c23fec9e7145eb P 8337af1943564c0dad18b45a597b805a: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:58:58.356762 10693 raft_consensus.cc:482] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:58.356766 10843 raft_consensus.cc:348] T d3db04c54a184eb8a1c23fec9e7145eb P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:58.356843 10845 tablet_bootstrap.cc:492] T 12e3bfe638d249c0934feda41b8a8876 P 6cc5b9962ed24441bcba0c07b9f5438e: No bootstrap required, opened a new log
I20240502 07:58:58.356868 10843 raft_consensus.cc:374] T d3db04c54a184eb8a1c23fec9e7145eb P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:58.356913 10845 ts_tablet_manager.cc:1367] T 12e3bfe638d249c0934feda41b8a8876 P 6cc5b9962ed24441bcba0c07b9f5438e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:58.356868 10693 raft_consensus.cc:504] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:58.356931 10843 raft_consensus.cc:729] T d3db04c54a184eb8a1c23fec9e7145eb P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8337af1943564c0dad18b45a597b805a, State: Initialized, Role: FOLLOWER
I20240502 07:58:58.357049 10843 consensus_queue.cc:260] T d3db04c54a184eb8a1c23fec9e7145eb P 8337af1943564c0dad18b45a597b805a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:58.357225 10693 leader_election.cc:288] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191), 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379)
I20240502 07:58:58.357316 10845 raft_consensus.cc:348] T 12e3bfe638d249c0934feda41b8a8876 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:58.357380 10843 ts_tablet_manager.cc:1398] T d3db04c54a184eb8a1c23fec9e7145eb P 8337af1943564c0dad18b45a597b805a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:58.357399 10845 raft_consensus.cc:374] T 12e3bfe638d249c0934feda41b8a8876 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:58.357430 10845 raft_consensus.cc:729] T 12e3bfe638d249c0934feda41b8a8876 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cc5b9962ed24441bcba0c07b9f5438e, State: Initialized, Role: FOLLOWER
I20240502 07:58:58.357547 10845 consensus_queue.cc:260] T 12e3bfe638d249c0934feda41b8a8876 P 6cc5b9962ed24441bcba0c07b9f5438e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:58.357733 9777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecd0995c31fe4e708712c62ca60511c9" candidate_uuid: "8337af1943564c0dad18b45a597b805a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1472c64688914149a318c7b8850a3df3" is_pre_election: true
I20240502 07:58:58.357784 10845 ts_tablet_manager.cc:1398] T 12e3bfe638d249c0934feda41b8a8876 P 6cc5b9962ed24441bcba0c07b9f5438e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:58.357859 9777 raft_consensus.cc:2454] T ecd0995c31fe4e708712c62ca60511c9 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8337af1943564c0dad18b45a597b805a in term 0.
I20240502 07:58:58.358202 9908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecd0995c31fe4e708712c62ca60511c9" candidate_uuid: "8337af1943564c0dad18b45a597b805a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" is_pre_election: true
I20240502 07:58:58.358299 9582 leader_election.cc:302] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1472c64688914149a318c7b8850a3df3, 8337af1943564c0dad18b45a597b805a; no voters:
I20240502 07:58:58.358319 9908 raft_consensus.cc:2454] T ecd0995c31fe4e708712c62ca60511c9 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8337af1943564c0dad18b45a597b805a in term 0.
I20240502 07:58:58.358438 10693 raft_consensus.cc:2781] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:58.358498 10693 raft_consensus.cc:482] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:58.358530 10693 raft_consensus.cc:3037] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:58.359452 10693 raft_consensus.cc:504] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:58.359797 10693 leader_election.cc:288] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a [CANDIDATE]: Term 1 election: Requested vote from peers 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191), 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379)
I20240502 07:58:58.360078 9777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecd0995c31fe4e708712c62ca60511c9" candidate_uuid: "8337af1943564c0dad18b45a597b805a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1472c64688914149a318c7b8850a3df3"
I20240502 07:58:58.360226 9777 raft_consensus.cc:3037] T ecd0995c31fe4e708712c62ca60511c9 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:58.360227 9908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecd0995c31fe4e708712c62ca60511c9" candidate_uuid: "8337af1943564c0dad18b45a597b805a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cc5b9962ed24441bcba0c07b9f5438e"
I20240502 07:58:58.360325 9908 raft_consensus.cc:3037] T ecd0995c31fe4e708712c62ca60511c9 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:58.361094 9777 raft_consensus.cc:2454] T ecd0995c31fe4e708712c62ca60511c9 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8337af1943564c0dad18b45a597b805a in term 1.
I20240502 07:58:58.361094 9908 raft_consensus.cc:2454] T ecd0995c31fe4e708712c62ca60511c9 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8337af1943564c0dad18b45a597b805a in term 1.
I20240502 07:58:58.361418 9582 leader_election.cc:302] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1472c64688914149a318c7b8850a3df3, 8337af1943564c0dad18b45a597b805a; no voters:
I20240502 07:58:58.361615 10855 raft_consensus.cc:2781] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:58.361694 10855 raft_consensus.cc:686] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Becoming Leader. State: Replica: 8337af1943564c0dad18b45a597b805a, State: Running, Role: LEADER
I20240502 07:58:58.361819 10855 consensus_queue.cc:237] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:58.363173 9371 catalog_manager.cc:5484] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a reported cstate change: term changed from 0 to 1, leader changed from <none> to 8337af1943564c0dad18b45a597b805a (127.3.139.66). New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:58.370344 10855 raft_consensus.cc:482] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:58.370445 10855 raft_consensus.cc:504] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:58.370723 10855 leader_election.cc:288] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551), 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379)
I20240502 07:58:58.371073 9514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12e3bfe638d249c0934feda41b8a8876" candidate_uuid: "8337af1943564c0dad18b45a597b805a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" is_pre_election: true
I20240502 07:58:58.371160 9908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12e3bfe638d249c0934feda41b8a8876" candidate_uuid: "8337af1943564c0dad18b45a597b805a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" is_pre_election: true
I20240502 07:58:58.371234 9514 raft_consensus.cc:2454] T 12e3bfe638d249c0934feda41b8a8876 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8337af1943564c0dad18b45a597b805a in term 0.
I20240502 07:58:58.371264 9908 raft_consensus.cc:2454] T 12e3bfe638d249c0934feda41b8a8876 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8337af1943564c0dad18b45a597b805a in term 0.
I20240502 07:58:58.371553 9582 leader_election.cc:302] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8337af1943564c0dad18b45a597b805a, a201db721bda4dc3a4422dc8fa9bf72e; no voters:
I20240502 07:58:58.371702 10855 raft_consensus.cc:2781] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:58.371766 10855 raft_consensus.cc:482] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:58.371803 10855 raft_consensus.cc:3037] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:58.411904 10855 raft_consensus.cc:504] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:58.412310 10855 leader_election.cc:288] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [CANDIDATE]: Term 1 election: Requested vote from peers a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551), 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379)
I20240502 07:58:58.412746 9908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12e3bfe638d249c0934feda41b8a8876" candidate_uuid: "8337af1943564c0dad18b45a597b805a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cc5b9962ed24441bcba0c07b9f5438e"
I20240502 07:58:58.412770 9514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12e3bfe638d249c0934feda41b8a8876" candidate_uuid: "8337af1943564c0dad18b45a597b805a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a201db721bda4dc3a4422dc8fa9bf72e"
I20240502 07:58:58.412878 9908 raft_consensus.cc:3037] T 12e3bfe638d249c0934feda41b8a8876 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:58.412899 9514 raft_consensus.cc:3037] T 12e3bfe638d249c0934feda41b8a8876 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:58.413659 9514 raft_consensus.cc:2454] T 12e3bfe638d249c0934feda41b8a8876 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8337af1943564c0dad18b45a597b805a in term 1.
I20240502 07:58:58.413694 9908 raft_consensus.cc:2454] T 12e3bfe638d249c0934feda41b8a8876 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8337af1943564c0dad18b45a597b805a in term 1.
I20240502 07:58:58.413951 9582 leader_election.cc:302] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8337af1943564c0dad18b45a597b805a, a201db721bda4dc3a4422dc8fa9bf72e; no voters:
I20240502 07:58:58.414124 10855 raft_consensus.cc:2781] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:58.414211 10855 raft_consensus.cc:686] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Becoming Leader. State: Replica: 8337af1943564c0dad18b45a597b805a, State: Running, Role: LEADER
I20240502 07:58:58.414353 10855 consensus_queue.cc:237] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:58.416101 9364 catalog_manager.cc:5484] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a reported cstate change: term changed from 0 to 1, leader changed from <none> to 8337af1943564c0dad18b45a597b805a (127.3.139.66). New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:58.678016 10766 raft_consensus.cc:482] T d3db04c54a184eb8a1c23fec9e7145eb P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:58.678133 10766 raft_consensus.cc:504] T d3db04c54a184eb8a1c23fec9e7145eb P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:58.678531 10766 leader_election.cc:288] T d3db04c54a184eb8a1c23fec9e7145eb P 1472c64688914149a318c7b8850a3df3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379), 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667)
I20240502 07:58:58.678915 9908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3db04c54a184eb8a1c23fec9e7145eb" candidate_uuid: "1472c64688914149a318c7b8850a3df3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" is_pre_election: true
I20240502 07:58:58.678934 9646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3db04c54a184eb8a1c23fec9e7145eb" candidate_uuid: "1472c64688914149a318c7b8850a3df3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8337af1943564c0dad18b45a597b805a" is_pre_election: true
I20240502 07:58:58.679073 9908 raft_consensus.cc:2454] T d3db04c54a184eb8a1c23fec9e7145eb P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1472c64688914149a318c7b8850a3df3 in term 0.
I20240502 07:58:58.679077 9646 raft_consensus.cc:2454] T d3db04c54a184eb8a1c23fec9e7145eb P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1472c64688914149a318c7b8850a3df3 in term 0.
I20240502 07:58:58.679368 9711 leader_election.cc:302] T d3db04c54a184eb8a1c23fec9e7145eb P 1472c64688914149a318c7b8850a3df3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1472c64688914149a318c7b8850a3df3, 8337af1943564c0dad18b45a597b805a; no voters:
I20240502 07:58:58.679541 10766 raft_consensus.cc:2781] T d3db04c54a184eb8a1c23fec9e7145eb P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:58.679598 10766 raft_consensus.cc:482] T d3db04c54a184eb8a1c23fec9e7145eb P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:58.679625 10766 raft_consensus.cc:3037] T d3db04c54a184eb8a1c23fec9e7145eb P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:58.680485 10766 raft_consensus.cc:504] T d3db04c54a184eb8a1c23fec9e7145eb P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:58.680780 10766 leader_election.cc:288] T d3db04c54a184eb8a1c23fec9e7145eb P 1472c64688914149a318c7b8850a3df3 [CANDIDATE]: Term 1 election: Requested vote from peers 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379), 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667)
I20240502 07:58:58.681092 9908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3db04c54a184eb8a1c23fec9e7145eb" candidate_uuid: "1472c64688914149a318c7b8850a3df3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cc5b9962ed24441bcba0c07b9f5438e"
I20240502 07:58:58.681192 9908 raft_consensus.cc:3037] T d3db04c54a184eb8a1c23fec9e7145eb P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:58.681174 9646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3db04c54a184eb8a1c23fec9e7145eb" candidate_uuid: "1472c64688914149a318c7b8850a3df3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8337af1943564c0dad18b45a597b805a"
I20240502 07:58:58.681280 9646 raft_consensus.cc:3037] T d3db04c54a184eb8a1c23fec9e7145eb P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:58.681896 9908 raft_consensus.cc:2454] T d3db04c54a184eb8a1c23fec9e7145eb P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1472c64688914149a318c7b8850a3df3 in term 1.
I20240502 07:58:58.681959 9646 raft_consensus.cc:2454] T d3db04c54a184eb8a1c23fec9e7145eb P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1472c64688914149a318c7b8850a3df3 in term 1.
I20240502 07:58:58.682161 9711 leader_election.cc:302] T d3db04c54a184eb8a1c23fec9e7145eb P 1472c64688914149a318c7b8850a3df3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1472c64688914149a318c7b8850a3df3, 6cc5b9962ed24441bcba0c07b9f5438e; no voters:
I20240502 07:58:58.682307 10766 raft_consensus.cc:2781] T d3db04c54a184eb8a1c23fec9e7145eb P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:58.682466 10766 raft_consensus.cc:686] T d3db04c54a184eb8a1c23fec9e7145eb P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Becoming Leader. State: Replica: 1472c64688914149a318c7b8850a3df3, State: Running, Role: LEADER
I20240502 07:58:58.682593 10766 consensus_queue.cc:237] T d3db04c54a184eb8a1c23fec9e7145eb P 1472c64688914149a318c7b8850a3df3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:58.684150 9364 catalog_manager.cc:5484] T d3db04c54a184eb8a1c23fec9e7145eb P 1472c64688914149a318c7b8850a3df3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1472c64688914149a318c7b8850a3df3 (127.3.139.67). New cstate: current_term: 1 leader_uuid: "1472c64688914149a318c7b8850a3df3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: HEALTHY } } }
I20240502 07:58:58.767905 10855 consensus_queue.cc:1035] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:58.771260 10693 consensus_queue.cc:1035] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 07:58:58.809880 10860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:58.810091 10860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:58.813128 10860 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_5'
I20240502 07:58:58.896322 9906 raft_consensus.cc:1261] T d3db04c54a184eb8a1c23fec9e7145eb P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:58.896684 9907 raft_consensus.cc:1261] T 12e3bfe638d249c0934feda41b8a8876 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:58.897333 9514 raft_consensus.cc:1261] T 12e3bfe638d249c0934feda41b8a8876 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:58.898550 10693 consensus_queue.cc:1035] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:58.899227 9646 raft_consensus.cc:1261] T d3db04c54a184eb8a1c23fec9e7145eb P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:58.899695 10891 consensus_queue.cc:1035] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:58.899683 10858 consensus_queue.cc:1035] T d3db04c54a184eb8a1c23fec9e7145eb P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:58.900535 10858 consensus_queue.cc:1035] T d3db04c54a184eb8a1c23fec9e7145eb P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:58.919180 10823 mvcc.cc:205] Tried to move back new op lower bound from 7023152082514296832 to 7023152081645150208. Current Snapshot: MvccSnapshot[applied={T|T < 7023152082514296832}]
INSERT report
rows total: 2000
time total: 118.047 ms
time per row: 0.0590234 ms
I20240502 07:58:58.968459 9370 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47034:
name: "rebalance_test_table_6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:58:58.973553 9493 tablet_service.cc:1467] Processing CreateTablet for tablet 364d227472894712a2ae15f4589a5a5b (DEFAULT_TABLE table=rebalance_test_table_6 [id=af6c4a01744d4fcfbdf53cfc8d19500b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:58.973628 9626 tablet_service.cc:1467] Processing CreateTablet for tablet 364d227472894712a2ae15f4589a5a5b (DEFAULT_TABLE table=rebalance_test_table_6 [id=af6c4a01744d4fcfbdf53cfc8d19500b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:58.973866 9626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 364d227472894712a2ae15f4589a5a5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:58.973867 9493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 364d227472894712a2ae15f4589a5a5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:58.974277 9888 tablet_service.cc:1467] Processing CreateTablet for tablet af07c527fff94115a743df1c73c3f4b5 (DEFAULT_TABLE table=rebalance_test_table_6 [id=af6c4a01744d4fcfbdf53cfc8d19500b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:58.974525 9888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af07c527fff94115a743df1c73c3f4b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:58.976198 10915 tablet_bootstrap.cc:492] T 364d227472894712a2ae15f4589a5a5b P a201db721bda4dc3a4422dc8fa9bf72e: Bootstrap starting.
I20240502 07:58:58.976827 9495 tablet_service.cc:1467] Processing CreateTablet for tablet 8cf1917de461431d98c5155db3eb2312 (DEFAULT_TABLE table=rebalance_test_table_6 [id=af6c4a01744d4fcfbdf53cfc8d19500b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:58.977192 10917 tablet_bootstrap.cc:492] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e: Bootstrap starting.
I20240502 07:58:58.977327 9755 tablet_service.cc:1467] Processing CreateTablet for tablet 8cf1917de461431d98c5155db3eb2312 (DEFAULT_TABLE table=rebalance_test_table_6 [id=af6c4a01744d4fcfbdf53cfc8d19500b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:58.977321 9757 tablet_service.cc:1467] Processing CreateTablet for tablet af07c527fff94115a743df1c73c3f4b5 (DEFAULT_TABLE table=rebalance_test_table_6 [id=af6c4a01744d4fcfbdf53cfc8d19500b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:58.976822 9493 tablet_service.cc:1467] Processing CreateTablet for tablet af07c527fff94115a743df1c73c3f4b5 (DEFAULT_TABLE table=rebalance_test_table_6 [id=af6c4a01744d4fcfbdf53cfc8d19500b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:58.977599 9755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cf1917de461431d98c5155db3eb2312. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:58.977712 9757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af07c527fff94115a743df1c73c3f4b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:58.977067 9495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cf1917de461431d98c5155db3eb2312. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:58.978037 10917 tablet_bootstrap.cc:654] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:58.977320 9756 tablet_service.cc:1467] Processing CreateTablet for tablet 364d227472894712a2ae15f4589a5a5b (DEFAULT_TABLE table=rebalance_test_table_6 [id=af6c4a01744d4fcfbdf53cfc8d19500b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:58.978765 9756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 364d227472894712a2ae15f4589a5a5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:58.979157 9493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af07c527fff94115a743df1c73c3f4b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:58.977061 10915 tablet_bootstrap.cc:654] T 364d227472894712a2ae15f4589a5a5b P a201db721bda4dc3a4422dc8fa9bf72e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:58.980633 10915 tablet_bootstrap.cc:492] T 364d227472894712a2ae15f4589a5a5b P a201db721bda4dc3a4422dc8fa9bf72e: No bootstrap required, opened a new log
I20240502 07:58:58.980741 10915 ts_tablet_manager.cc:1367] T 364d227472894712a2ae15f4589a5a5b P a201db721bda4dc3a4422dc8fa9bf72e: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:58:58.981119 10917 tablet_bootstrap.cc:492] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e: No bootstrap required, opened a new log
I20240502 07:58:58.981200 10917 ts_tablet_manager.cc:1367] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:58:58.981175 10915 raft_consensus.cc:348] T 364d227472894712a2ae15f4589a5a5b P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:58.981268 10915 raft_consensus.cc:374] T 364d227472894712a2ae15f4589a5a5b P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:58.981295 10915 raft_consensus.cc:729] T 364d227472894712a2ae15f4589a5a5b P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a201db721bda4dc3a4422dc8fa9bf72e, State: Initialized, Role: FOLLOWER
I20240502 07:58:58.981410 10915 consensus_queue.cc:260] T 364d227472894712a2ae15f4589a5a5b P a201db721bda4dc3a4422dc8fa9bf72e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:58.981571 10917 raft_consensus.cc:348] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:58.981665 10917 raft_consensus.cc:374] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:58.981673 10915 ts_tablet_manager.cc:1398] T 364d227472894712a2ae15f4589a5a5b P a201db721bda4dc3a4422dc8fa9bf72e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:58.981694 10917 raft_consensus.cc:729] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cc5b9962ed24441bcba0c07b9f5438e, State: Initialized, Role: FOLLOWER
I20240502 07:58:58.981838 10915 tablet_bootstrap.cc:492] T 8cf1917de461431d98c5155db3eb2312 P a201db721bda4dc3a4422dc8fa9bf72e: Bootstrap starting.
I20240502 07:58:58.981799 10917 consensus_queue.cc:260] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:58.982072 10917 ts_tablet_manager.cc:1398] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:58.982225 9626 tablet_service.cc:1467] Processing CreateTablet for tablet 8cf1917de461431d98c5155db3eb2312 (DEFAULT_TABLE table=rebalance_test_table_6 [id=af6c4a01744d4fcfbdf53cfc8d19500b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:58.982457 9626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cf1917de461431d98c5155db3eb2312. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:58.983104 10915 tablet_bootstrap.cc:654] T 8cf1917de461431d98c5155db3eb2312 P a201db721bda4dc3a4422dc8fa9bf72e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:58.984444 10915 tablet_bootstrap.cc:492] T 8cf1917de461431d98c5155db3eb2312 P a201db721bda4dc3a4422dc8fa9bf72e: No bootstrap required, opened a new log
I20240502 07:58:58.984687 10915 ts_tablet_manager.cc:1367] T 8cf1917de461431d98c5155db3eb2312 P a201db721bda4dc3a4422dc8fa9bf72e: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 07:58:58.985141 10915 raft_consensus.cc:348] T 8cf1917de461431d98c5155db3eb2312 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:58.985252 10915 raft_consensus.cc:374] T 8cf1917de461431d98c5155db3eb2312 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:58.985299 10915 raft_consensus.cc:729] T 8cf1917de461431d98c5155db3eb2312 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a201db721bda4dc3a4422dc8fa9bf72e, State: Initialized, Role: FOLLOWER
I20240502 07:58:58.985412 10915 consensus_queue.cc:260] T 8cf1917de461431d98c5155db3eb2312 P a201db721bda4dc3a4422dc8fa9bf72e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:58.985670 10916 tablet_bootstrap.cc:492] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a: Bootstrap starting.
I20240502 07:58:58.985792 10915 ts_tablet_manager.cc:1398] T 8cf1917de461431d98c5155db3eb2312 P a201db721bda4dc3a4422dc8fa9bf72e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:58.986006 10915 tablet_bootstrap.cc:492] T af07c527fff94115a743df1c73c3f4b5 P a201db721bda4dc3a4422dc8fa9bf72e: Bootstrap starting.
I20240502 07:58:58.986831 10915 tablet_bootstrap.cc:654] T af07c527fff94115a743df1c73c3f4b5 P a201db721bda4dc3a4422dc8fa9bf72e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:58.987195 10916 tablet_bootstrap.cc:654] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:58.987974 10919 tablet_bootstrap.cc:492] T 8cf1917de461431d98c5155db3eb2312 P 1472c64688914149a318c7b8850a3df3: Bootstrap starting.
I20240502 07:58:58.988428 10915 tablet_bootstrap.cc:492] T af07c527fff94115a743df1c73c3f4b5 P a201db721bda4dc3a4422dc8fa9bf72e: No bootstrap required, opened a new log
I20240502 07:58:58.988512 10915 ts_tablet_manager.cc:1367] T af07c527fff94115a743df1c73c3f4b5 P a201db721bda4dc3a4422dc8fa9bf72e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 07:58:58.988687 10916 tablet_bootstrap.cc:492] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a: No bootstrap required, opened a new log
I20240502 07:58:58.988765 10916 ts_tablet_manager.cc:1367] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:58.988912 10915 raft_consensus.cc:348] T af07c527fff94115a743df1c73c3f4b5 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:58.989017 10915 raft_consensus.cc:374] T af07c527fff94115a743df1c73c3f4b5 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:58.989058 10919 tablet_bootstrap.cc:654] T 8cf1917de461431d98c5155db3eb2312 P 1472c64688914149a318c7b8850a3df3: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:58.989095 10915 raft_consensus.cc:729] T af07c527fff94115a743df1c73c3f4b5 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a201db721bda4dc3a4422dc8fa9bf72e, State: Initialized, Role: FOLLOWER
I20240502 07:58:58.989157 10916 raft_consensus.cc:348] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:58.989259 10916 raft_consensus.cc:374] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:58.989226 10915 consensus_queue.cc:260] T af07c527fff94115a743df1c73c3f4b5 P a201db721bda4dc3a4422dc8fa9bf72e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:58.989334 10916 raft_consensus.cc:729] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8337af1943564c0dad18b45a597b805a, State: Initialized, Role: FOLLOWER
I20240502 07:58:58.989463 10916 consensus_queue.cc:260] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:58.989763 10916 ts_tablet_manager.cc:1398] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:58.989966 10916 tablet_bootstrap.cc:492] T 8cf1917de461431d98c5155db3eb2312 P 8337af1943564c0dad18b45a597b805a: Bootstrap starting.
I20240502 07:58:58.989924 10915 ts_tablet_manager.cc:1398] T af07c527fff94115a743df1c73c3f4b5 P a201db721bda4dc3a4422dc8fa9bf72e: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240502 07:58:58.991016 10916 tablet_bootstrap.cc:654] T 8cf1917de461431d98c5155db3eb2312 P 8337af1943564c0dad18b45a597b805a: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:58.992128 10919 tablet_bootstrap.cc:492] T 8cf1917de461431d98c5155db3eb2312 P 1472c64688914149a318c7b8850a3df3: No bootstrap required, opened a new log
I20240502 07:58:58.992214 10919 ts_tablet_manager.cc:1367] T 8cf1917de461431d98c5155db3eb2312 P 1472c64688914149a318c7b8850a3df3: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 07:58:58.992482 10916 tablet_bootstrap.cc:492] T 8cf1917de461431d98c5155db3eb2312 P 8337af1943564c0dad18b45a597b805a: No bootstrap required, opened a new log
I20240502 07:58:58.992569 10916 ts_tablet_manager.cc:1367] T 8cf1917de461431d98c5155db3eb2312 P 8337af1943564c0dad18b45a597b805a: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240502 07:58:58.992703 10919 raft_consensus.cc:348] T 8cf1917de461431d98c5155db3eb2312 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:58.992800 10919 raft_consensus.cc:374] T 8cf1917de461431d98c5155db3eb2312 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:58.992839 10919 raft_consensus.cc:729] T 8cf1917de461431d98c5155db3eb2312 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1472c64688914149a318c7b8850a3df3, State: Initialized, Role: FOLLOWER
I20240502 07:58:58.992959 10919 consensus_queue.cc:260] T 8cf1917de461431d98c5155db3eb2312 P 1472c64688914149a318c7b8850a3df3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:58.992965 10916 raft_consensus.cc:348] T 8cf1917de461431d98c5155db3eb2312 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:58.993043 10916 raft_consensus.cc:374] T 8cf1917de461431d98c5155db3eb2312 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:58.993081 10916 raft_consensus.cc:729] T 8cf1917de461431d98c5155db3eb2312 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8337af1943564c0dad18b45a597b805a, State: Initialized, Role: FOLLOWER
I20240502 07:58:58.993193 10919 ts_tablet_manager.cc:1398] T 8cf1917de461431d98c5155db3eb2312 P 1472c64688914149a318c7b8850a3df3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:58.993194 10916 consensus_queue.cc:260] T 8cf1917de461431d98c5155db3eb2312 P 8337af1943564c0dad18b45a597b805a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:58.993402 10919 tablet_bootstrap.cc:492] T 364d227472894712a2ae15f4589a5a5b P 1472c64688914149a318c7b8850a3df3: Bootstrap starting.
I20240502 07:58:58.993449 10916 ts_tablet_manager.cc:1398] T 8cf1917de461431d98c5155db3eb2312 P 8337af1943564c0dad18b45a597b805a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:58.994143 10919 tablet_bootstrap.cc:654] T 364d227472894712a2ae15f4589a5a5b P 1472c64688914149a318c7b8850a3df3: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:58.994617 10782 raft_consensus.cc:482] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:58.994705 10782 raft_consensus.cc:504] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:58.995098 10782 leader_election.cc:288] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191), a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551)
I20240502 07:58:58.995342 9777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af07c527fff94115a743df1c73c3f4b5" candidate_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1472c64688914149a318c7b8850a3df3" is_pre_election: true
I20240502 07:58:58.995404 9514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af07c527fff94115a743df1c73c3f4b5" candidate_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" is_pre_election: true
I20240502 07:58:58.995509 9514 raft_consensus.cc:2454] T af07c527fff94115a743df1c73c3f4b5 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cc5b9962ed24441bcba0c07b9f5438e in term 0.
I20240502 07:58:58.995770 9844 leader_election.cc:302] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cc5b9962ed24441bcba0c07b9f5438e, a201db721bda4dc3a4422dc8fa9bf72e; no voters:
I20240502 07:58:58.995919 10782 raft_consensus.cc:2781] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:58.995963 10782 raft_consensus.cc:482] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:58.995993 10782 raft_consensus.cc:3037] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Advancing to term 1
W20240502 07:58:58.995991 9844 leader_election.cc:341] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191): Illegal state: must be running to vote when last-logged opid is not known
I20240502 07:58:58.996829 10919 tablet_bootstrap.cc:492] T 364d227472894712a2ae15f4589a5a5b P 1472c64688914149a318c7b8850a3df3: No bootstrap required, opened a new log
I20240502 07:58:58.996856 10782 raft_consensus.cc:504] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:58.996913 10919 ts_tablet_manager.cc:1367] T 364d227472894712a2ae15f4589a5a5b P 1472c64688914149a318c7b8850a3df3: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:58:58.997196 10782 leader_election.cc:288] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e [CANDIDATE]: Term 1 election: Requested vote from peers 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191), a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551)
I20240502 07:58:58.997437 9777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af07c527fff94115a743df1c73c3f4b5" candidate_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1472c64688914149a318c7b8850a3df3"
I20240502 07:58:58.997478 9514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af07c527fff94115a743df1c73c3f4b5" candidate_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a201db721bda4dc3a4422dc8fa9bf72e"
I20240502 07:58:58.997553 9514 raft_consensus.cc:3037] T af07c527fff94115a743df1c73c3f4b5 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:58.997313 10919 raft_consensus.cc:348] T 364d227472894712a2ae15f4589a5a5b P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:58.997648 10919 raft_consensus.cc:374] T 364d227472894712a2ae15f4589a5a5b P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:58.997689 10919 raft_consensus.cc:729] T 364d227472894712a2ae15f4589a5a5b P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1472c64688914149a318c7b8850a3df3, State: Initialized, Role: FOLLOWER
W20240502 07:58:58.997834 9844 leader_election.cc:341] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191): Illegal state: must be running to vote when last-logged opid is not known
I20240502 07:58:58.997810 10919 consensus_queue.cc:260] T 364d227472894712a2ae15f4589a5a5b P 1472c64688914149a318c7b8850a3df3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:58.998119 10919 ts_tablet_manager.cc:1398] T 364d227472894712a2ae15f4589a5a5b P 1472c64688914149a318c7b8850a3df3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:58.998339 10919 tablet_bootstrap.cc:492] T af07c527fff94115a743df1c73c3f4b5 P 1472c64688914149a318c7b8850a3df3: Bootstrap starting.
I20240502 07:58:58.998354 9514 raft_consensus.cc:2454] T af07c527fff94115a743df1c73c3f4b5 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cc5b9962ed24441bcba0c07b9f5438e in term 1.
I20240502 07:58:58.998661 9844 leader_election.cc:302] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6cc5b9962ed24441bcba0c07b9f5438e, a201db721bda4dc3a4422dc8fa9bf72e; no voters: 1472c64688914149a318c7b8850a3df3
I20240502 07:58:58.998924 10782 raft_consensus.cc:2781] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:58.999104 10782 raft_consensus.cc:686] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 LEADER]: Becoming Leader. State: Replica: 6cc5b9962ed24441bcba0c07b9f5438e, State: Running, Role: LEADER
I20240502 07:58:58.999212 10919 tablet_bootstrap.cc:654] T af07c527fff94115a743df1c73c3f4b5 P 1472c64688914149a318c7b8850a3df3: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:58.999262 10782 consensus_queue.cc:237] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:59.000496 10919 tablet_bootstrap.cc:492] T af07c527fff94115a743df1c73c3f4b5 P 1472c64688914149a318c7b8850a3df3: No bootstrap required, opened a new log
I20240502 07:58:59.000579 10919 ts_tablet_manager.cc:1367] T af07c527fff94115a743df1c73c3f4b5 P 1472c64688914149a318c7b8850a3df3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:59.000885 10919 raft_consensus.cc:348] T af07c527fff94115a743df1c73c3f4b5 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:59.000952 10919 raft_consensus.cc:374] T af07c527fff94115a743df1c73c3f4b5 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:59.000969 10919 raft_consensus.cc:729] T af07c527fff94115a743df1c73c3f4b5 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1472c64688914149a318c7b8850a3df3, State: Initialized, Role: FOLLOWER
I20240502 07:58:59.001056 10919 consensus_queue.cc:260] T af07c527fff94115a743df1c73c3f4b5 P 1472c64688914149a318c7b8850a3df3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:59.001259 10919 ts_tablet_manager.cc:1398] T af07c527fff94115a743df1c73c3f4b5 P 1472c64688914149a318c7b8850a3df3: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:59.001351 9370 catalog_manager.cc:5484] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68). New cstate: current_term: 1 leader_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: HEALTHY } } }
I20240502 07:58:59.017637 10511 raft_consensus.cc:482] T 8cf1917de461431d98c5155db3eb2312 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:59.017731 10511 raft_consensus.cc:504] T 8cf1917de461431d98c5155db3eb2312 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:59.018049 10511 leader_election.cc:288] T 8cf1917de461431d98c5155db3eb2312 P a201db721bda4dc3a4422dc8fa9bf72e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191), 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667)
I20240502 07:58:59.018379 9777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cf1917de461431d98c5155db3eb2312" candidate_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1472c64688914149a318c7b8850a3df3" is_pre_election: true
I20240502 07:58:59.018381 9646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cf1917de461431d98c5155db3eb2312" candidate_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8337af1943564c0dad18b45a597b805a" is_pre_election: true
I20240502 07:58:59.018491 9646 raft_consensus.cc:2454] T 8cf1917de461431d98c5155db3eb2312 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a201db721bda4dc3a4422dc8fa9bf72e in term 0.
I20240502 07:58:59.018491 9777 raft_consensus.cc:2454] T 8cf1917de461431d98c5155db3eb2312 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a201db721bda4dc3a4422dc8fa9bf72e in term 0.
I20240502 07:58:59.018719 9451 leader_election.cc:302] T 8cf1917de461431d98c5155db3eb2312 P a201db721bda4dc3a4422dc8fa9bf72e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1472c64688914149a318c7b8850a3df3, a201db721bda4dc3a4422dc8fa9bf72e; no voters:
I20240502 07:58:59.018895 10511 raft_consensus.cc:2781] T 8cf1917de461431d98c5155db3eb2312 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:59.018954 10511 raft_consensus.cc:482] T 8cf1917de461431d98c5155db3eb2312 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:59.018982 10511 raft_consensus.cc:3037] T 8cf1917de461431d98c5155db3eb2312 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:59.019685 10511 raft_consensus.cc:504] T 8cf1917de461431d98c5155db3eb2312 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:59.019986 10511 leader_election.cc:288] T 8cf1917de461431d98c5155db3eb2312 P a201db721bda4dc3a4422dc8fa9bf72e [CANDIDATE]: Term 1 election: Requested vote from peers 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191), 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667)
I20240502 07:58:59.020262 9777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cf1917de461431d98c5155db3eb2312" candidate_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1472c64688914149a318c7b8850a3df3"
I20240502 07:58:59.020308 9646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cf1917de461431d98c5155db3eb2312" candidate_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8337af1943564c0dad18b45a597b805a"
I20240502 07:58:59.020363 9777 raft_consensus.cc:3037] T 8cf1917de461431d98c5155db3eb2312 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:59.020395 9646 raft_consensus.cc:3037] T 8cf1917de461431d98c5155db3eb2312 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:59.021054 9777 raft_consensus.cc:2454] T 8cf1917de461431d98c5155db3eb2312 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a201db721bda4dc3a4422dc8fa9bf72e in term 1.
I20240502 07:58:59.021056 9646 raft_consensus.cc:2454] T 8cf1917de461431d98c5155db3eb2312 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a201db721bda4dc3a4422dc8fa9bf72e in term 1.
I20240502 07:58:59.021309 9451 leader_election.cc:302] T 8cf1917de461431d98c5155db3eb2312 P a201db721bda4dc3a4422dc8fa9bf72e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1472c64688914149a318c7b8850a3df3, a201db721bda4dc3a4422dc8fa9bf72e; no voters:
I20240502 07:58:59.021502 10511 raft_consensus.cc:2781] T 8cf1917de461431d98c5155db3eb2312 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:59.021687 10511 raft_consensus.cc:686] T 8cf1917de461431d98c5155db3eb2312 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Becoming Leader. State: Replica: a201db721bda4dc3a4422dc8fa9bf72e, State: Running, Role: LEADER
I20240502 07:58:59.021834 10511 consensus_queue.cc:237] T 8cf1917de461431d98c5155db3eb2312 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:59.023409 9364 catalog_manager.cc:5484] T 8cf1917de461431d98c5155db3eb2312 P a201db721bda4dc3a4422dc8fa9bf72e reported cstate change: term changed from 0 to 1, leader changed from <none> to a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65). New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:59.165496 10891 raft_consensus.cc:482] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:59.165637 10891 raft_consensus.cc:504] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:59.166038 10891 leader_election.cc:288] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551), 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191)
I20240502 07:58:59.166428 9514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "364d227472894712a2ae15f4589a5a5b" candidate_uuid: "8337af1943564c0dad18b45a597b805a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" is_pre_election: true
I20240502 07:58:59.166486 9777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "364d227472894712a2ae15f4589a5a5b" candidate_uuid: "8337af1943564c0dad18b45a597b805a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1472c64688914149a318c7b8850a3df3" is_pre_election: true
I20240502 07:58:59.166559 9514 raft_consensus.cc:2454] T 364d227472894712a2ae15f4589a5a5b P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8337af1943564c0dad18b45a597b805a in term 0.
I20240502 07:58:59.166628 9777 raft_consensus.cc:2454] T 364d227472894712a2ae15f4589a5a5b P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8337af1943564c0dad18b45a597b805a in term 0.
I20240502 07:58:59.166858 9582 leader_election.cc:302] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8337af1943564c0dad18b45a597b805a, a201db721bda4dc3a4422dc8fa9bf72e; no voters:
I20240502 07:58:59.167022 10891 raft_consensus.cc:2781] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:59.167106 10891 raft_consensus.cc:482] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:59.167152 10891 raft_consensus.cc:3037] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:59.167963 10891 raft_consensus.cc:504] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:59.168280 10891 leader_election.cc:288] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a [CANDIDATE]: Term 1 election: Requested vote from peers a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551), 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191)
I20240502 07:58:59.168697 9777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "364d227472894712a2ae15f4589a5a5b" candidate_uuid: "8337af1943564c0dad18b45a597b805a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1472c64688914149a318c7b8850a3df3"
I20240502 07:58:59.168717 9514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "364d227472894712a2ae15f4589a5a5b" candidate_uuid: "8337af1943564c0dad18b45a597b805a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a201db721bda4dc3a4422dc8fa9bf72e"
I20240502 07:58:59.168814 9777 raft_consensus.cc:3037] T 364d227472894712a2ae15f4589a5a5b P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:59.168829 9514 raft_consensus.cc:3037] T 364d227472894712a2ae15f4589a5a5b P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:59.169528 9514 raft_consensus.cc:2454] T 364d227472894712a2ae15f4589a5a5b P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8337af1943564c0dad18b45a597b805a in term 1.
I20240502 07:58:59.169608 9777 raft_consensus.cc:2454] T 364d227472894712a2ae15f4589a5a5b P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8337af1943564c0dad18b45a597b805a in term 1.
I20240502 07:58:59.169806 9582 leader_election.cc:302] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8337af1943564c0dad18b45a597b805a, a201db721bda4dc3a4422dc8fa9bf72e; no voters:
I20240502 07:58:59.169965 10891 raft_consensus.cc:2781] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:59.170068 10891 raft_consensus.cc:686] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Becoming Leader. State: Replica: 8337af1943564c0dad18b45a597b805a, State: Running, Role: LEADER
I20240502 07:58:59.170212 10891 consensus_queue.cc:237] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:59.171917 9364 catalog_manager.cc:5484] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a reported cstate change: term changed from 0 to 1, leader changed from <none> to 8337af1943564c0dad18b45a597b805a (127.3.139.66). New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:59.300977 10766 raft_consensus.cc:482] T af07c527fff94115a743df1c73c3f4b5 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:59.301106 10766 raft_consensus.cc:504] T af07c527fff94115a743df1c73c3f4b5 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } }
I20240502 07:58:59.301532 10766 leader_election.cc:288] T af07c527fff94115a743df1c73c3f4b5 P 1472c64688914149a318c7b8850a3df3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551), 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379)
I20240502 07:58:59.301848 9906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af07c527fff94115a743df1c73c3f4b5" candidate_uuid: "1472c64688914149a318c7b8850a3df3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" is_pre_election: true
I20240502 07:58:59.301857 9514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af07c527fff94115a743df1c73c3f4b5" candidate_uuid: "1472c64688914149a318c7b8850a3df3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" is_pre_election: true
I20240502 07:58:59.302019 9514 raft_consensus.cc:2379] T af07c527fff94115a743df1c73c3f4b5 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1472c64688914149a318c7b8850a3df3 in current term 1: Already voted for candidate 6cc5b9962ed24441bcba0c07b9f5438e in this term.
I20240502 07:58:59.302348 9713 leader_election.cc:302] T af07c527fff94115a743df1c73c3f4b5 P 1472c64688914149a318c7b8850a3df3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1472c64688914149a318c7b8850a3df3; no voters: 6cc5b9962ed24441bcba0c07b9f5438e, a201db721bda4dc3a4422dc8fa9bf72e
I20240502 07:58:59.302568 10766 raft_consensus.cc:3037] T af07c527fff94115a743df1c73c3f4b5 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:59.303409 10766 raft_consensus.cc:2726] T af07c527fff94115a743df1c73c3f4b5 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20240502 07:58:59.319173 10933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:59.319370 10933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:59.322248 10933 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_6'
I20240502 07:58:59.394476 9514 raft_consensus.cc:1261] T af07c527fff94115a743df1c73c3f4b5 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:59.394896 10782 consensus_queue.cc:1035] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:59.395380 9777 raft_consensus.cc:1261] T af07c527fff94115a743df1c73c3f4b5 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:59.395855 10782 consensus_queue.cc:1035] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:59.396597 9777 raft_consensus.cc:1261] T 8cf1917de461431d98c5155db3eb2312 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:59.396840 9646 raft_consensus.cc:1261] T 8cf1917de461431d98c5155db3eb2312 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:59.397208 10511 consensus_queue.cc:1035] T 8cf1917de461431d98c5155db3eb2312 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:59.397586 10511 consensus_queue.cc:1035] T 8cf1917de461431d98c5155db3eb2312 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:59.416457 10890 mvcc.cc:205] Tried to move back new op lower bound from 7023152084564430848 to 7023152083034783744. Current Snapshot: MvccSnapshot[applied={T|T < 7023152084564430848}]
I20240502 07:58:59.432206 9514 raft_consensus.cc:1261] T 364d227472894712a2ae15f4589a5a5b P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:59.432718 9776 raft_consensus.cc:1261] T 364d227472894712a2ae15f4589a5a5b P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:59.432709 10693 consensus_queue.cc:1035] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:59.433056 10693 consensus_queue.cc:1035] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 101.369 ms
time per row: 0.0506845 ms
I20240502 07:58:59.464444 9364 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47034:
name: "rebalance_test_table_7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:58:59.469414 9888 tablet_service.cc:1467] Processing CreateTablet for tablet f7335ac0209c4e28b6751bb01d34d665 (DEFAULT_TABLE table=rebalance_test_table_7 [id=7100a319a6ec42499621cff56501ef8f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:59.469591 9626 tablet_service.cc:1467] Processing CreateTablet for tablet f7335ac0209c4e28b6751bb01d34d665 (DEFAULT_TABLE table=rebalance_test_table_7 [id=7100a319a6ec42499621cff56501ef8f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:59.469728 9888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7335ac0209c4e28b6751bb01d34d665. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:59.469878 9626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7335ac0209c4e28b6751bb01d34d665. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:59.471350 10917 tablet_bootstrap.cc:492] T f7335ac0209c4e28b6751bb01d34d665 P 6cc5b9962ed24441bcba0c07b9f5438e: Bootstrap starting.
I20240502 07:58:59.471603 9493 tablet_service.cc:1467] Processing CreateTablet for tablet f7335ac0209c4e28b6751bb01d34d665 (DEFAULT_TABLE table=rebalance_test_table_7 [id=7100a319a6ec42499621cff56501ef8f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:59.471801 10916 tablet_bootstrap.cc:492] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a: Bootstrap starting.
I20240502 07:58:59.471859 9493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7335ac0209c4e28b6751bb01d34d665. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:59.471943 9757 tablet_service.cc:1467] Processing CreateTablet for tablet 265eca6fff644d83857ed48a04864a58 (DEFAULT_TABLE table=rebalance_test_table_7 [id=7100a319a6ec42499621cff56501ef8f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:59.472190 9757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 265eca6fff644d83857ed48a04864a58. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:59.472191 10917 tablet_bootstrap.cc:654] T f7335ac0209c4e28b6751bb01d34d665 P 6cc5b9962ed24441bcba0c07b9f5438e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:59.472514 10916 tablet_bootstrap.cc:654] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:59.473570 9495 tablet_service.cc:1467] Processing CreateTablet for tablet 265eca6fff644d83857ed48a04864a58 (DEFAULT_TABLE table=rebalance_test_table_7 [id=7100a319a6ec42499621cff56501ef8f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:59.473702 9756 tablet_service.cc:1467] Processing CreateTablet for tablet 427c037fa09c4b94b7bc6c71bdb61c2f (DEFAULT_TABLE table=rebalance_test_table_7 [id=7100a319a6ec42499621cff56501ef8f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:59.473793 9495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 265eca6fff644d83857ed48a04864a58. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:59.473924 9756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 427c037fa09c4b94b7bc6c71bdb61c2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:59.473951 9625 tablet_service.cc:1467] Processing CreateTablet for tablet 427c037fa09c4b94b7bc6c71bdb61c2f (DEFAULT_TABLE table=rebalance_test_table_7 [id=7100a319a6ec42499621cff56501ef8f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:59.474152 9625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 427c037fa09c4b94b7bc6c71bdb61c2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:59.474320 10915 tablet_bootstrap.cc:492] T f7335ac0209c4e28b6751bb01d34d665 P a201db721bda4dc3a4422dc8fa9bf72e: Bootstrap starting.
I20240502 07:58:59.475131 10915 tablet_bootstrap.cc:654] T f7335ac0209c4e28b6751bb01d34d665 P a201db721bda4dc3a4422dc8fa9bf72e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:59.475750 10919 tablet_bootstrap.cc:492] T 265eca6fff644d83857ed48a04864a58 P 1472c64688914149a318c7b8850a3df3: Bootstrap starting.
I20240502 07:58:59.476329 9888 tablet_service.cc:1467] Processing CreateTablet for tablet 265eca6fff644d83857ed48a04864a58 (DEFAULT_TABLE table=rebalance_test_table_7 [id=7100a319a6ec42499621cff56501ef8f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:59.476486 10915 tablet_bootstrap.cc:492] T f7335ac0209c4e28b6751bb01d34d665 P a201db721bda4dc3a4422dc8fa9bf72e: No bootstrap required, opened a new log
I20240502 07:58:59.476536 10919 tablet_bootstrap.cc:654] T 265eca6fff644d83857ed48a04864a58 P 1472c64688914149a318c7b8850a3df3: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:59.476552 9888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 265eca6fff644d83857ed48a04864a58. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:59.476562 10915 ts_tablet_manager.cc:1367] T f7335ac0209c4e28b6751bb01d34d665 P a201db721bda4dc3a4422dc8fa9bf72e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 07:58:59.476853 10917 tablet_bootstrap.cc:492] T f7335ac0209c4e28b6751bb01d34d665 P 6cc5b9962ed24441bcba0c07b9f5438e: No bootstrap required, opened a new log
I20240502 07:58:59.476922 10917 ts_tablet_manager.cc:1367] T f7335ac0209c4e28b6751bb01d34d665 P 6cc5b9962ed24441bcba0c07b9f5438e: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240502 07:58:59.476996 10915 raft_consensus.cc:348] T f7335ac0209c4e28b6751bb01d34d665 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:59.477092 10915 raft_consensus.cc:374] T f7335ac0209c4e28b6751bb01d34d665 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:59.477121 10915 raft_consensus.cc:729] T f7335ac0209c4e28b6751bb01d34d665 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a201db721bda4dc3a4422dc8fa9bf72e, State: Initialized, Role: FOLLOWER
I20240502 07:58:59.477237 10915 consensus_queue.cc:260] T f7335ac0209c4e28b6751bb01d34d665 P a201db721bda4dc3a4422dc8fa9bf72e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:59.477277 10917 raft_consensus.cc:348] T f7335ac0209c4e28b6751bb01d34d665 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:59.477385 10917 raft_consensus.cc:374] T f7335ac0209c4e28b6751bb01d34d665 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:59.477418 10917 raft_consensus.cc:729] T f7335ac0209c4e28b6751bb01d34d665 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cc5b9962ed24441bcba0c07b9f5438e, State: Initialized, Role: FOLLOWER
I20240502 07:58:59.477535 10916 tablet_bootstrap.cc:492] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a: No bootstrap required, opened a new log
I20240502 07:58:59.477530 10915 ts_tablet_manager.cc:1398] T f7335ac0209c4e28b6751bb01d34d665 P a201db721bda4dc3a4422dc8fa9bf72e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:59.477527 10917 consensus_queue.cc:260] T f7335ac0209c4e28b6751bb01d34d665 P 6cc5b9962ed24441bcba0c07b9f5438e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:59.477603 10916 ts_tablet_manager.cc:1367] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20240502 07:58:59.477684 10915 tablet_bootstrap.cc:492] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e: Bootstrap starting.
I20240502 07:58:59.477777 10917 ts_tablet_manager.cc:1398] T f7335ac0209c4e28b6751bb01d34d665 P 6cc5b9962ed24441bcba0c07b9f5438e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:59.478005 10916 raft_consensus.cc:348] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:59.478099 10916 raft_consensus.cc:374] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:59.478127 10916 raft_consensus.cc:729] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8337af1943564c0dad18b45a597b805a, State: Initialized, Role: FOLLOWER
I20240502 07:58:59.476329 9886 tablet_service.cc:1467] Processing CreateTablet for tablet 427c037fa09c4b94b7bc6c71bdb61c2f (DEFAULT_TABLE table=rebalance_test_table_7 [id=7100a319a6ec42499621cff56501ef8f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:59.478241 10916 consensus_queue.cc:260] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:59.478432 9886 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 427c037fa09c4b94b7bc6c71bdb61c2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:59.478497 10915 tablet_bootstrap.cc:654] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:59.478494 10916 ts_tablet_manager.cc:1398] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240502 07:58:59.478648 10916 tablet_bootstrap.cc:492] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a: Bootstrap starting.
I20240502 07:58:59.479463 10916 tablet_bootstrap.cc:654] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:59.480578 10915 tablet_bootstrap.cc:492] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e: No bootstrap required, opened a new log
I20240502 07:58:59.480655 10915 ts_tablet_manager.cc:1367] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:59.481068 10915 raft_consensus.cc:348] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:58:59.481185 10915 raft_consensus.cc:374] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:59.481227 10915 raft_consensus.cc:729] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a201db721bda4dc3a4422dc8fa9bf72e, State: Initialized, Role: FOLLOWER
I20240502 07:58:59.481352 10915 consensus_queue.cc:260] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:58:59.481534 10919 tablet_bootstrap.cc:492] T 265eca6fff644d83857ed48a04864a58 P 1472c64688914149a318c7b8850a3df3: No bootstrap required, opened a new log
I20240502 07:58:59.481602 10915 ts_tablet_manager.cc:1398] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:59.481617 10919 ts_tablet_manager.cc:1367] T 265eca6fff644d83857ed48a04864a58 P 1472c64688914149a318c7b8850a3df3: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240502 07:58:59.482024 10919 raft_consensus.cc:348] T 265eca6fff644d83857ed48a04864a58 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:58:59.482128 10919 raft_consensus.cc:374] T 265eca6fff644d83857ed48a04864a58 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:59.482158 10919 raft_consensus.cc:729] T 265eca6fff644d83857ed48a04864a58 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1472c64688914149a318c7b8850a3df3, State: Initialized, Role: FOLLOWER
I20240502 07:58:59.482265 10919 consensus_queue.cc:260] T 265eca6fff644d83857ed48a04864a58 P 1472c64688914149a318c7b8850a3df3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:58:59.482528 10919 ts_tablet_manager.cc:1398] T 265eca6fff644d83857ed48a04864a58 P 1472c64688914149a318c7b8850a3df3: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:59.482694 10919 tablet_bootstrap.cc:492] T 427c037fa09c4b94b7bc6c71bdb61c2f P 1472c64688914149a318c7b8850a3df3: Bootstrap starting.
I20240502 07:58:59.483543 10919 tablet_bootstrap.cc:654] T 427c037fa09c4b94b7bc6c71bdb61c2f P 1472c64688914149a318c7b8850a3df3: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:59.483999 10916 tablet_bootstrap.cc:492] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a: No bootstrap required, opened a new log
I20240502 07:58:59.484084 10916 ts_tablet_manager.cc:1367] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:58:59.484500 10916 raft_consensus.cc:348] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:59.484604 10916 raft_consensus.cc:374] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:59.484642 10916 raft_consensus.cc:729] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8337af1943564c0dad18b45a597b805a, State: Initialized, Role: FOLLOWER
I20240502 07:58:59.484760 10916 consensus_queue.cc:260] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:59.484822 10919 tablet_bootstrap.cc:492] T 427c037fa09c4b94b7bc6c71bdb61c2f P 1472c64688914149a318c7b8850a3df3: No bootstrap required, opened a new log
I20240502 07:58:59.485020 10916 ts_tablet_manager.cc:1398] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:59.485113 10919 ts_tablet_manager.cc:1367] T 427c037fa09c4b94b7bc6c71bdb61c2f P 1472c64688914149a318c7b8850a3df3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:59.485534 10919 raft_consensus.cc:348] T 427c037fa09c4b94b7bc6c71bdb61c2f P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:59.485644 10919 raft_consensus.cc:374] T 427c037fa09c4b94b7bc6c71bdb61c2f P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:59.485683 10919 raft_consensus.cc:729] T 427c037fa09c4b94b7bc6c71bdb61c2f P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1472c64688914149a318c7b8850a3df3, State: Initialized, Role: FOLLOWER
I20240502 07:58:59.485797 10919 consensus_queue.cc:260] T 427c037fa09c4b94b7bc6c71bdb61c2f P 1472c64688914149a318c7b8850a3df3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:59.486059 10919 ts_tablet_manager.cc:1398] T 427c037fa09c4b94b7bc6c71bdb61c2f P 1472c64688914149a318c7b8850a3df3: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:59.486106 10917 tablet_bootstrap.cc:492] T 427c037fa09c4b94b7bc6c71bdb61c2f P 6cc5b9962ed24441bcba0c07b9f5438e: Bootstrap starting.
I20240502 07:58:59.486922 10917 tablet_bootstrap.cc:654] T 427c037fa09c4b94b7bc6c71bdb61c2f P 6cc5b9962ed24441bcba0c07b9f5438e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:59.488157 10917 tablet_bootstrap.cc:492] T 427c037fa09c4b94b7bc6c71bdb61c2f P 6cc5b9962ed24441bcba0c07b9f5438e: No bootstrap required, opened a new log
I20240502 07:58:59.488225 10917 ts_tablet_manager.cc:1367] T 427c037fa09c4b94b7bc6c71bdb61c2f P 6cc5b9962ed24441bcba0c07b9f5438e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:59.488610 10917 raft_consensus.cc:348] T 427c037fa09c4b94b7bc6c71bdb61c2f P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:59.488690 10917 raft_consensus.cc:374] T 427c037fa09c4b94b7bc6c71bdb61c2f P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:59.488720 10917 raft_consensus.cc:729] T 427c037fa09c4b94b7bc6c71bdb61c2f P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cc5b9962ed24441bcba0c07b9f5438e, State: Initialized, Role: FOLLOWER
I20240502 07:58:59.488837 10917 consensus_queue.cc:260] T 427c037fa09c4b94b7bc6c71bdb61c2f P 6cc5b9962ed24441bcba0c07b9f5438e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:59.489044 10917 ts_tablet_manager.cc:1398] T 427c037fa09c4b94b7bc6c71bdb61c2f P 6cc5b9962ed24441bcba0c07b9f5438e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:59.489204 10917 tablet_bootstrap.cc:492] T 265eca6fff644d83857ed48a04864a58 P 6cc5b9962ed24441bcba0c07b9f5438e: Bootstrap starting.
I20240502 07:58:59.489914 10917 tablet_bootstrap.cc:654] T 265eca6fff644d83857ed48a04864a58 P 6cc5b9962ed24441bcba0c07b9f5438e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:59.490869 10917 tablet_bootstrap.cc:492] T 265eca6fff644d83857ed48a04864a58 P 6cc5b9962ed24441bcba0c07b9f5438e: No bootstrap required, opened a new log
I20240502 07:58:59.490942 10917 ts_tablet_manager.cc:1367] T 265eca6fff644d83857ed48a04864a58 P 6cc5b9962ed24441bcba0c07b9f5438e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:59.491340 10917 raft_consensus.cc:348] T 265eca6fff644d83857ed48a04864a58 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:58:59.491437 10917 raft_consensus.cc:374] T 265eca6fff644d83857ed48a04864a58 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:59.491466 10917 raft_consensus.cc:729] T 265eca6fff644d83857ed48a04864a58 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cc5b9962ed24441bcba0c07b9f5438e, State: Initialized, Role: FOLLOWER
I20240502 07:58:59.491626 10917 consensus_queue.cc:260] T 265eca6fff644d83857ed48a04864a58 P 6cc5b9962ed24441bcba0c07b9f5438e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:58:59.491863 10917 ts_tablet_manager.cc:1398] T 265eca6fff644d83857ed48a04864a58 P 6cc5b9962ed24441bcba0c07b9f5438e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:59.642627 10891 raft_consensus.cc:482] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:59.642741 10891 raft_consensus.cc:504] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:59.643121 10891 leader_election.cc:288] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379), a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551)
I20240502 07:58:59.643464 9907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7335ac0209c4e28b6751bb01d34d665" candidate_uuid: "8337af1943564c0dad18b45a597b805a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" is_pre_election: true
I20240502 07:58:59.643486 9514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7335ac0209c4e28b6751bb01d34d665" candidate_uuid: "8337af1943564c0dad18b45a597b805a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" is_pre_election: true
I20240502 07:58:59.643582 9907 raft_consensus.cc:2454] T f7335ac0209c4e28b6751bb01d34d665 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8337af1943564c0dad18b45a597b805a in term 0.
I20240502 07:58:59.643623 9514 raft_consensus.cc:2454] T f7335ac0209c4e28b6751bb01d34d665 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8337af1943564c0dad18b45a597b805a in term 0.
I20240502 07:58:59.643875 9580 leader_election.cc:302] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cc5b9962ed24441bcba0c07b9f5438e, 8337af1943564c0dad18b45a597b805a; no voters:
I20240502 07:58:59.644042 10891 raft_consensus.cc:2781] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:59.644100 10891 raft_consensus.cc:482] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:59.644126 10891 raft_consensus.cc:3037] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:59.644925 10891 raft_consensus.cc:504] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:59.645236 10891 leader_election.cc:288] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a [CANDIDATE]: Term 1 election: Requested vote from peers 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379), a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551)
I20240502 07:58:59.645567 9514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7335ac0209c4e28b6751bb01d34d665" candidate_uuid: "8337af1943564c0dad18b45a597b805a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a201db721bda4dc3a4422dc8fa9bf72e"
I20240502 07:58:59.645574 9907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7335ac0209c4e28b6751bb01d34d665" candidate_uuid: "8337af1943564c0dad18b45a597b805a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cc5b9962ed24441bcba0c07b9f5438e"
I20240502 07:58:59.645673 9514 raft_consensus.cc:3037] T f7335ac0209c4e28b6751bb01d34d665 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:59.645681 9907 raft_consensus.cc:3037] T f7335ac0209c4e28b6751bb01d34d665 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:59.646523 9907 raft_consensus.cc:2454] T f7335ac0209c4e28b6751bb01d34d665 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8337af1943564c0dad18b45a597b805a in term 1.
I20240502 07:58:59.646654 9514 raft_consensus.cc:2454] T f7335ac0209c4e28b6751bb01d34d665 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8337af1943564c0dad18b45a597b805a in term 1.
I20240502 07:58:59.646764 9580 leader_election.cc:302] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cc5b9962ed24441bcba0c07b9f5438e, 8337af1943564c0dad18b45a597b805a; no voters:
I20240502 07:58:59.646927 10693 raft_consensus.cc:2781] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:59.647008 10693 raft_consensus.cc:686] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Becoming Leader. State: Replica: 8337af1943564c0dad18b45a597b805a, State: Running, Role: LEADER
I20240502 07:58:59.647162 10693 consensus_queue.cc:237] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:58:59.648752 9364 catalog_manager.cc:5484] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a reported cstate change: term changed from 0 to 1, leader changed from <none> to 8337af1943564c0dad18b45a597b805a (127.3.139.66). New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: HEALTHY } } }
I20240502 07:58:59.755098 10693 raft_consensus.cc:482] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:59.755209 10693 raft_consensus.cc:504] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:59.755568 10693 leader_election.cc:288] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379), 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191)
I20240502 07:58:59.755909 9777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "427c037fa09c4b94b7bc6c71bdb61c2f" candidate_uuid: "8337af1943564c0dad18b45a597b805a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1472c64688914149a318c7b8850a3df3" is_pre_election: true
I20240502 07:58:59.756038 9777 raft_consensus.cc:2454] T 427c037fa09c4b94b7bc6c71bdb61c2f P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8337af1943564c0dad18b45a597b805a in term 0.
I20240502 07:58:59.756204 9907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "427c037fa09c4b94b7bc6c71bdb61c2f" candidate_uuid: "8337af1943564c0dad18b45a597b805a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" is_pre_election: true
I20240502 07:58:59.756299 9582 leader_election.cc:302] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1472c64688914149a318c7b8850a3df3, 8337af1943564c0dad18b45a597b805a; no voters:
I20240502 07:58:59.756341 9907 raft_consensus.cc:2454] T 427c037fa09c4b94b7bc6c71bdb61c2f P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8337af1943564c0dad18b45a597b805a in term 0.
I20240502 07:58:59.756543 10693 raft_consensus.cc:2781] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:59.756594 10693 raft_consensus.cc:482] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:59.756629 10693 raft_consensus.cc:3037] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:59.757586 10693 raft_consensus.cc:504] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:59.757905 10693 leader_election.cc:288] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a [CANDIDATE]: Term 1 election: Requested vote from peers 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379), 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191)
I20240502 07:58:59.758188 9777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "427c037fa09c4b94b7bc6c71bdb61c2f" candidate_uuid: "8337af1943564c0dad18b45a597b805a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1472c64688914149a318c7b8850a3df3"
I20240502 07:58:59.758291 9777 raft_consensus.cc:3037] T 427c037fa09c4b94b7bc6c71bdb61c2f P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:59.758282 9907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "427c037fa09c4b94b7bc6c71bdb61c2f" candidate_uuid: "8337af1943564c0dad18b45a597b805a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cc5b9962ed24441bcba0c07b9f5438e"
I20240502 07:58:59.758385 9907 raft_consensus.cc:3037] T 427c037fa09c4b94b7bc6c71bdb61c2f P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:59.759189 9777 raft_consensus.cc:2454] T 427c037fa09c4b94b7bc6c71bdb61c2f P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8337af1943564c0dad18b45a597b805a in term 1.
I20240502 07:58:59.759250 9907 raft_consensus.cc:2454] T 427c037fa09c4b94b7bc6c71bdb61c2f P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8337af1943564c0dad18b45a597b805a in term 1.
I20240502 07:58:59.759471 9582 leader_election.cc:302] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1472c64688914149a318c7b8850a3df3, 8337af1943564c0dad18b45a597b805a; no voters:
I20240502 07:58:59.759627 10693 raft_consensus.cc:2781] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:59.759716 10693 raft_consensus.cc:686] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Becoming Leader. State: Replica: 8337af1943564c0dad18b45a597b805a, State: Running, Role: LEADER
I20240502 07:58:59.759845 10693 consensus_queue.cc:237] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:58:59.763041 9364 catalog_manager.cc:5484] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a reported cstate change: term changed from 0 to 1, leader changed from <none> to 8337af1943564c0dad18b45a597b805a (127.3.139.66). New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: UNKNOWN } } }
I20240502 07:58:59.817274 10930 raft_consensus.cc:482] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:59.817390 10930 raft_consensus.cc:504] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:58:59.817727 10930 leader_election.cc:288] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191), 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379)
I20240502 07:58:59.818032 9777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "265eca6fff644d83857ed48a04864a58" candidate_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1472c64688914149a318c7b8850a3df3" is_pre_election: true
I20240502 07:58:59.818054 9907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "265eca6fff644d83857ed48a04864a58" candidate_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" is_pre_election: true
I20240502 07:58:59.818152 9777 raft_consensus.cc:2454] T 265eca6fff644d83857ed48a04864a58 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a201db721bda4dc3a4422dc8fa9bf72e in term 0.
I20240502 07:58:59.818190 9907 raft_consensus.cc:2454] T 265eca6fff644d83857ed48a04864a58 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a201db721bda4dc3a4422dc8fa9bf72e in term 0.
I20240502 07:58:59.818393 9451 leader_election.cc:302] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1472c64688914149a318c7b8850a3df3, a201db721bda4dc3a4422dc8fa9bf72e; no voters:
I20240502 07:58:59.818583 10930 raft_consensus.cc:2781] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:59.818634 10930 raft_consensus.cc:482] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:59.818660 10930 raft_consensus.cc:3037] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:59.819512 10930 raft_consensus.cc:504] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:58:59.819820 10930 leader_election.cc:288] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e [CANDIDATE]: Term 1 election: Requested vote from peers 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191), 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379)
I20240502 07:58:59.820082 9777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "265eca6fff644d83857ed48a04864a58" candidate_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1472c64688914149a318c7b8850a3df3"
I20240502 07:58:59.820119 9907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "265eca6fff644d83857ed48a04864a58" candidate_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cc5b9962ed24441bcba0c07b9f5438e"
I20240502 07:58:59.820169 9777 raft_consensus.cc:3037] T 265eca6fff644d83857ed48a04864a58 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:59.820228 9907 raft_consensus.cc:3037] T 265eca6fff644d83857ed48a04864a58 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:59.820982 9777 raft_consensus.cc:2454] T 265eca6fff644d83857ed48a04864a58 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a201db721bda4dc3a4422dc8fa9bf72e in term 1.
I20240502 07:58:59.821202 9451 leader_election.cc:302] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1472c64688914149a318c7b8850a3df3, a201db721bda4dc3a4422dc8fa9bf72e; no voters:
I20240502 07:58:59.821220 9907 raft_consensus.cc:2454] T 265eca6fff644d83857ed48a04864a58 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a201db721bda4dc3a4422dc8fa9bf72e in term 1.
I20240502 07:58:59.821497 10930 raft_consensus.cc:2781] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:59.821573 10930 raft_consensus.cc:686] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Becoming Leader. State: Replica: a201db721bda4dc3a4422dc8fa9bf72e, State: Running, Role: LEADER
I20240502 07:58:59.821699 10930 consensus_queue.cc:237] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:58:59.823330 9364 catalog_manager.cc:5484] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e reported cstate change: term changed from 0 to 1, leader changed from <none> to a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65). New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: HEALTHY } } }
W20240502 07:58:59.940343 10987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:59.940547 10987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:59.943562 10987 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_7'
I20240502 07:59:00.021188 9907 raft_consensus.cc:1261] T f7335ac0209c4e28b6751bb01d34d665 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:00.021507 9906 raft_consensus.cc:1261] T 427c037fa09c4b94b7bc6c71bdb61c2f P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:00.021760 9514 raft_consensus.cc:1261] T f7335ac0209c4e28b6751bb01d34d665 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:00.022136 9777 raft_consensus.cc:1261] T 427c037fa09c4b94b7bc6c71bdb61c2f P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:00.022490 10891 consensus_queue.cc:1035] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:00.022979 10891 consensus_queue.cc:1035] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:00.024432 10693 consensus_queue.cc:1035] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:00.024708 10693 consensus_queue.cc:1035] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:00.025568 9776 raft_consensus.cc:1261] T 265eca6fff644d83857ed48a04864a58 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:00.025905 10930 consensus_queue.cc:1035] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:00.026772 9908 raft_consensus.cc:1261] T 265eca6fff644d83857ed48a04864a58 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:00.030515 10930 consensus_queue.cc:1035] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 98.9599 ms
time per row: 0.0494799 ms
I20240502 07:59:00.081614 9372 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47034:
name: "rebalance_test_table_8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:59:00.087322 9625 tablet_service.cc:1467] Processing CreateTablet for tablet fd251de859e049c38a632a17b2fbdeae (DEFAULT_TABLE table=rebalance_test_table_8 [id=6aecdf32373e4c9db62c5a4844dc38ca]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:00.087647 9625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd251de859e049c38a632a17b2fbdeae. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:00.089411 11049 tablet_bootstrap.cc:492] T fd251de859e049c38a632a17b2fbdeae P 8337af1943564c0dad18b45a597b805a: Bootstrap starting.
I20240502 07:59:00.089573 9888 tablet_service.cc:1467] Processing CreateTablet for tablet fd251de859e049c38a632a17b2fbdeae (DEFAULT_TABLE table=rebalance_test_table_8 [id=6aecdf32373e4c9db62c5a4844dc38ca]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:00.089819 9888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd251de859e049c38a632a17b2fbdeae. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:00.090191 11049 tablet_bootstrap.cc:654] T fd251de859e049c38a632a17b2fbdeae P 8337af1943564c0dad18b45a597b805a: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:00.091523 9756 tablet_service.cc:1467] Processing CreateTablet for tablet bc896a65284c48cd864bb6b57dbe5ca0 (DEFAULT_TABLE table=rebalance_test_table_8 [id=6aecdf32373e4c9db62c5a4844dc38ca]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:00.091697 9886 tablet_service.cc:1467] Processing CreateTablet for tablet 0e4efe560ef74763b80b70665c11d7f1 (DEFAULT_TABLE table=rebalance_test_table_8 [id=6aecdf32373e4c9db62c5a4844dc38ca]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:00.091775 9756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc896a65284c48cd864bb6b57dbe5ca0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:00.091908 9886 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e4efe560ef74763b80b70665c11d7f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:00.091899 9757 tablet_service.cc:1467] Processing CreateTablet for tablet 0e4efe560ef74763b80b70665c11d7f1 (DEFAULT_TABLE table=rebalance_test_table_8 [id=6aecdf32373e4c9db62c5a4844dc38ca]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:00.092087 9757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e4efe560ef74763b80b70665c11d7f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:00.093808 11052 tablet_bootstrap.cc:492] T bc896a65284c48cd864bb6b57dbe5ca0 P 1472c64688914149a318c7b8850a3df3: Bootstrap starting.
I20240502 07:59:00.094548 9495 tablet_service.cc:1467] Processing CreateTablet for tablet fd251de859e049c38a632a17b2fbdeae (DEFAULT_TABLE table=rebalance_test_table_8 [id=6aecdf32373e4c9db62c5a4844dc38ca]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:00.094635 11052 tablet_bootstrap.cc:654] T bc896a65284c48cd864bb6b57dbe5ca0 P 1472c64688914149a318c7b8850a3df3: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:00.094799 9495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd251de859e049c38a632a17b2fbdeae. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:00.095332 9494 tablet_service.cc:1467] Processing CreateTablet for tablet 0e4efe560ef74763b80b70665c11d7f1 (DEFAULT_TABLE table=rebalance_test_table_8 [id=6aecdf32373e4c9db62c5a4844dc38ca]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:00.095496 9494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e4efe560ef74763b80b70665c11d7f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:00.096916 9493 tablet_service.cc:1467] Processing CreateTablet for tablet bc896a65284c48cd864bb6b57dbe5ca0 (DEFAULT_TABLE table=rebalance_test_table_8 [id=6aecdf32373e4c9db62c5a4844dc38ca]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:00.097108 9493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc896a65284c48cd864bb6b57dbe5ca0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:00.097013 11054 tablet_bootstrap.cc:492] T fd251de859e049c38a632a17b2fbdeae P a201db721bda4dc3a4422dc8fa9bf72e: Bootstrap starting.
I20240502 07:59:00.097229 11051 tablet_bootstrap.cc:492] T fd251de859e049c38a632a17b2fbdeae P 6cc5b9962ed24441bcba0c07b9f5438e: Bootstrap starting.
I20240502 07:59:00.097955 11051 tablet_bootstrap.cc:654] T fd251de859e049c38a632a17b2fbdeae P 6cc5b9962ed24441bcba0c07b9f5438e: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:00.097972 11054 tablet_bootstrap.cc:654] T fd251de859e049c38a632a17b2fbdeae P a201db721bda4dc3a4422dc8fa9bf72e: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:00.098946 11052 tablet_bootstrap.cc:492] T bc896a65284c48cd864bb6b57dbe5ca0 P 1472c64688914149a318c7b8850a3df3: No bootstrap required, opened a new log
I20240502 07:59:00.099032 11052 ts_tablet_manager.cc:1367] T bc896a65284c48cd864bb6b57dbe5ca0 P 1472c64688914149a318c7b8850a3df3: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20240502 07:59:00.099099 11051 tablet_bootstrap.cc:492] T fd251de859e049c38a632a17b2fbdeae P 6cc5b9962ed24441bcba0c07b9f5438e: No bootstrap required, opened a new log
I20240502 07:59:00.099159 11051 ts_tablet_manager.cc:1367] T fd251de859e049c38a632a17b2fbdeae P 6cc5b9962ed24441bcba0c07b9f5438e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:00.099529 11052 raft_consensus.cc:348] T bc896a65284c48cd864bb6b57dbe5ca0 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:59:00.099635 11052 raft_consensus.cc:374] T bc896a65284c48cd864bb6b57dbe5ca0 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:00.099666 11052 raft_consensus.cc:729] T bc896a65284c48cd864bb6b57dbe5ca0 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1472c64688914149a318c7b8850a3df3, State: Initialized, Role: FOLLOWER
I20240502 07:59:00.099692 11051 raft_consensus.cc:348] T fd251de859e049c38a632a17b2fbdeae P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:59:00.099776 11051 raft_consensus.cc:374] T fd251de859e049c38a632a17b2fbdeae P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:00.099805 11051 raft_consensus.cc:729] T fd251de859e049c38a632a17b2fbdeae P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cc5b9962ed24441bcba0c07b9f5438e, State: Initialized, Role: FOLLOWER
I20240502 07:59:00.099781 11052 consensus_queue.cc:260] T bc896a65284c48cd864bb6b57dbe5ca0 P 1472c64688914149a318c7b8850a3df3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:59:00.099844 9625 tablet_service.cc:1467] Processing CreateTablet for tablet bc896a65284c48cd864bb6b57dbe5ca0 (DEFAULT_TABLE table=rebalance_test_table_8 [id=6aecdf32373e4c9db62c5a4844dc38ca]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:00.099926 11051 consensus_queue.cc:260] T fd251de859e049c38a632a17b2fbdeae P 6cc5b9962ed24441bcba0c07b9f5438e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:59:00.100039 11052 ts_tablet_manager.cc:1398] T bc896a65284c48cd864bb6b57dbe5ca0 P 1472c64688914149a318c7b8850a3df3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:00.100072 9625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc896a65284c48cd864bb6b57dbe5ca0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:00.100163 11051 ts_tablet_manager.cc:1398] T fd251de859e049c38a632a17b2fbdeae P 6cc5b9962ed24441bcba0c07b9f5438e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:00.100191 11052 tablet_bootstrap.cc:492] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3: Bootstrap starting.
I20240502 07:59:00.100324 11051 tablet_bootstrap.cc:492] T 0e4efe560ef74763b80b70665c11d7f1 P 6cc5b9962ed24441bcba0c07b9f5438e: Bootstrap starting.
I20240502 07:59:00.101001 11052 tablet_bootstrap.cc:654] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:00.101032 11051 tablet_bootstrap.cc:654] T 0e4efe560ef74763b80b70665c11d7f1 P 6cc5b9962ed24441bcba0c07b9f5438e: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:00.102489 11049 tablet_bootstrap.cc:492] T fd251de859e049c38a632a17b2fbdeae P 8337af1943564c0dad18b45a597b805a: No bootstrap required, opened a new log
I20240502 07:59:00.102573 11049 ts_tablet_manager.cc:1367] T fd251de859e049c38a632a17b2fbdeae P 8337af1943564c0dad18b45a597b805a: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.000s
I20240502 07:59:00.102941 11049 raft_consensus.cc:348] T fd251de859e049c38a632a17b2fbdeae P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:59:00.103040 11049 raft_consensus.cc:374] T fd251de859e049c38a632a17b2fbdeae P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:00.103096 11049 raft_consensus.cc:729] T fd251de859e049c38a632a17b2fbdeae P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8337af1943564c0dad18b45a597b805a, State: Initialized, Role: FOLLOWER
I20240502 07:59:00.103220 11049 consensus_queue.cc:260] T fd251de859e049c38a632a17b2fbdeae P 8337af1943564c0dad18b45a597b805a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:59:00.103452 11049 ts_tablet_manager.cc:1398] T fd251de859e049c38a632a17b2fbdeae P 8337af1943564c0dad18b45a597b805a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:00.103605 11049 tablet_bootstrap.cc:492] T bc896a65284c48cd864bb6b57dbe5ca0 P 8337af1943564c0dad18b45a597b805a: Bootstrap starting.
I20240502 07:59:00.104298 11049 tablet_bootstrap.cc:654] T bc896a65284c48cd864bb6b57dbe5ca0 P 8337af1943564c0dad18b45a597b805a: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:00.105093 11054 tablet_bootstrap.cc:492] T fd251de859e049c38a632a17b2fbdeae P a201db721bda4dc3a4422dc8fa9bf72e: No bootstrap required, opened a new log
I20240502 07:59:00.105185 11054 ts_tablet_manager.cc:1367] T fd251de859e049c38a632a17b2fbdeae P a201db721bda4dc3a4422dc8fa9bf72e: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240502 07:59:00.105202 11049 tablet_bootstrap.cc:492] T bc896a65284c48cd864bb6b57dbe5ca0 P 8337af1943564c0dad18b45a597b805a: No bootstrap required, opened a new log
I20240502 07:59:00.105304 11049 ts_tablet_manager.cc:1367] T bc896a65284c48cd864bb6b57dbe5ca0 P 8337af1943564c0dad18b45a597b805a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 07:59:00.105603 11054 raft_consensus.cc:348] T fd251de859e049c38a632a17b2fbdeae P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:59:00.105710 11054 raft_consensus.cc:374] T fd251de859e049c38a632a17b2fbdeae P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:00.105681 11049 raft_consensus.cc:348] T bc896a65284c48cd864bb6b57dbe5ca0 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:59:00.105743 11054 raft_consensus.cc:729] T fd251de859e049c38a632a17b2fbdeae P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a201db721bda4dc3a4422dc8fa9bf72e, State: Initialized, Role: FOLLOWER
I20240502 07:59:00.105774 11049 raft_consensus.cc:374] T bc896a65284c48cd864bb6b57dbe5ca0 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:00.105810 11049 raft_consensus.cc:729] T bc896a65284c48cd864bb6b57dbe5ca0 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8337af1943564c0dad18b45a597b805a, State: Initialized, Role: FOLLOWER
I20240502 07:59:00.105852 11054 consensus_queue.cc:260] T fd251de859e049c38a632a17b2fbdeae P a201db721bda4dc3a4422dc8fa9bf72e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:59:00.105919 11049 consensus_queue.cc:260] T bc896a65284c48cd864bb6b57dbe5ca0 P 8337af1943564c0dad18b45a597b805a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:59:00.106112 11054 ts_tablet_manager.cc:1398] T fd251de859e049c38a632a17b2fbdeae P a201db721bda4dc3a4422dc8fa9bf72e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:00.106344 11049 ts_tablet_manager.cc:1398] T bc896a65284c48cd864bb6b57dbe5ca0 P 8337af1943564c0dad18b45a597b805a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:00.106806 11054 tablet_bootstrap.cc:492] T 0e4efe560ef74763b80b70665c11d7f1 P a201db721bda4dc3a4422dc8fa9bf72e: Bootstrap starting.
I20240502 07:59:00.107664 11054 tablet_bootstrap.cc:654] T 0e4efe560ef74763b80b70665c11d7f1 P a201db721bda4dc3a4422dc8fa9bf72e: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:00.108748 11054 tablet_bootstrap.cc:492] T 0e4efe560ef74763b80b70665c11d7f1 P a201db721bda4dc3a4422dc8fa9bf72e: No bootstrap required, opened a new log
I20240502 07:59:00.108834 11054 ts_tablet_manager.cc:1367] T 0e4efe560ef74763b80b70665c11d7f1 P a201db721bda4dc3a4422dc8fa9bf72e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:00.109015 11052 tablet_bootstrap.cc:492] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3: No bootstrap required, opened a new log
I20240502 07:59:00.109093 11052 ts_tablet_manager.cc:1367] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.001s
I20240502 07:59:00.109112 11051 tablet_bootstrap.cc:492] T 0e4efe560ef74763b80b70665c11d7f1 P 6cc5b9962ed24441bcba0c07b9f5438e: No bootstrap required, opened a new log
I20240502 07:59:00.109180 11051 ts_tablet_manager.cc:1367] T 0e4efe560ef74763b80b70665c11d7f1 P 6cc5b9962ed24441bcba0c07b9f5438e: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20240502 07:59:00.109207 11054 raft_consensus.cc:348] T 0e4efe560ef74763b80b70665c11d7f1 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:59:00.109285 11054 raft_consensus.cc:374] T 0e4efe560ef74763b80b70665c11d7f1 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:00.109314 11054 raft_consensus.cc:729] T 0e4efe560ef74763b80b70665c11d7f1 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a201db721bda4dc3a4422dc8fa9bf72e, State: Initialized, Role: FOLLOWER
I20240502 07:59:00.109421 11054 consensus_queue.cc:260] T 0e4efe560ef74763b80b70665c11d7f1 P a201db721bda4dc3a4422dc8fa9bf72e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:59:00.109493 11052 raft_consensus.cc:348] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:59:00.109581 11052 raft_consensus.cc:374] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:00.109609 11052 raft_consensus.cc:729] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1472c64688914149a318c7b8850a3df3, State: Initialized, Role: FOLLOWER
I20240502 07:59:00.109644 11054 ts_tablet_manager.cc:1398] T 0e4efe560ef74763b80b70665c11d7f1 P a201db721bda4dc3a4422dc8fa9bf72e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:00.109716 11052 consensus_queue.cc:260] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:59:00.109723 11051 raft_consensus.cc:348] T 0e4efe560ef74763b80b70665c11d7f1 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:59:00.109783 11054 tablet_bootstrap.cc:492] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e: Bootstrap starting.
I20240502 07:59:00.109992 11052 ts_tablet_manager.cc:1398] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:00.109808 11051 raft_consensus.cc:374] T 0e4efe560ef74763b80b70665c11d7f1 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:00.110397 11051 raft_consensus.cc:729] T 0e4efe560ef74763b80b70665c11d7f1 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cc5b9962ed24441bcba0c07b9f5438e, State: Initialized, Role: FOLLOWER
I20240502 07:59:00.110550 11054 tablet_bootstrap.cc:654] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:00.110550 11051 consensus_queue.cc:260] T 0e4efe560ef74763b80b70665c11d7f1 P 6cc5b9962ed24441bcba0c07b9f5438e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:59:00.111291 11051 ts_tablet_manager.cc:1398] T 0e4efe560ef74763b80b70665c11d7f1 P 6cc5b9962ed24441bcba0c07b9f5438e: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:00.112162 11054 tablet_bootstrap.cc:492] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e: No bootstrap required, opened a new log
I20240502 07:59:00.112234 11054 ts_tablet_manager.cc:1367] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:00.112600 11054 raft_consensus.cc:348] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:59:00.112696 11054 raft_consensus.cc:374] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:00.112733 11054 raft_consensus.cc:729] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a201db721bda4dc3a4422dc8fa9bf72e, State: Initialized, Role: FOLLOWER
I20240502 07:59:00.112847 11054 consensus_queue.cc:260] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:59:00.113157 11054 ts_tablet_manager.cc:1398] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:00.198038 10858 raft_consensus.cc:482] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:00.198169 10858 raft_consensus.cc:504] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:59:00.198588 10858 leader_election.cc:288] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379), a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551)
I20240502 07:59:00.198963 9906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4efe560ef74763b80b70665c11d7f1" candidate_uuid: "1472c64688914149a318c7b8850a3df3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" is_pre_election: true
I20240502 07:59:00.198983 9514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4efe560ef74763b80b70665c11d7f1" candidate_uuid: "1472c64688914149a318c7b8850a3df3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" is_pre_election: true
I20240502 07:59:00.199115 9906 raft_consensus.cc:2454] T 0e4efe560ef74763b80b70665c11d7f1 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1472c64688914149a318c7b8850a3df3 in term 0.
I20240502 07:59:00.199142 9514 raft_consensus.cc:2454] T 0e4efe560ef74763b80b70665c11d7f1 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1472c64688914149a318c7b8850a3df3 in term 0.
I20240502 07:59:00.199395 9711 leader_election.cc:302] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1472c64688914149a318c7b8850a3df3, 6cc5b9962ed24441bcba0c07b9f5438e; no voters:
I20240502 07:59:00.199569 10858 raft_consensus.cc:2781] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:00.199628 10858 raft_consensus.cc:482] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:00.199654 10858 raft_consensus.cc:3037] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:00.200448 10858 raft_consensus.cc:504] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:59:00.200747 10858 leader_election.cc:288] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 [CANDIDATE]: Term 1 election: Requested vote from peers 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379), a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551)
I20240502 07:59:00.201012 9906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4efe560ef74763b80b70665c11d7f1" candidate_uuid: "1472c64688914149a318c7b8850a3df3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cc5b9962ed24441bcba0c07b9f5438e"
I20240502 07:59:00.201050 9514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4efe560ef74763b80b70665c11d7f1" candidate_uuid: "1472c64688914149a318c7b8850a3df3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a201db721bda4dc3a4422dc8fa9bf72e"
I20240502 07:59:00.201144 9514 raft_consensus.cc:3037] T 0e4efe560ef74763b80b70665c11d7f1 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:00.201144 9906 raft_consensus.cc:3037] T 0e4efe560ef74763b80b70665c11d7f1 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:00.201826 9514 raft_consensus.cc:2454] T 0e4efe560ef74763b80b70665c11d7f1 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1472c64688914149a318c7b8850a3df3 in term 1.
I20240502 07:59:00.201893 9906 raft_consensus.cc:2454] T 0e4efe560ef74763b80b70665c11d7f1 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1472c64688914149a318c7b8850a3df3 in term 1.
I20240502 07:59:00.202121 9713 leader_election.cc:302] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1472c64688914149a318c7b8850a3df3, a201db721bda4dc3a4422dc8fa9bf72e; no voters:
I20240502 07:59:00.202275 10858 raft_consensus.cc:2781] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:00.202473 10858 raft_consensus.cc:686] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Becoming Leader. State: Replica: 1472c64688914149a318c7b8850a3df3, State: Running, Role: LEADER
I20240502 07:59:00.202600 10858 consensus_queue.cc:237] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:59:00.204246 9372 catalog_manager.cc:5484] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1472c64688914149a318c7b8850a3df3 (127.3.139.67). New cstate: current_term: 1 leader_uuid: "1472c64688914149a318c7b8850a3df3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:00.248346 10511 raft_consensus.cc:482] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:00.248456 10511 raft_consensus.cc:504] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:59:00.248849 10511 leader_election.cc:288] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667), 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191)
I20240502 07:59:00.249214 9777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc896a65284c48cd864bb6b57dbe5ca0" candidate_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1472c64688914149a318c7b8850a3df3" is_pre_election: true
I20240502 07:59:00.249234 9644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc896a65284c48cd864bb6b57dbe5ca0" candidate_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8337af1943564c0dad18b45a597b805a" is_pre_election: true
I20240502 07:59:00.249341 9777 raft_consensus.cc:2454] T bc896a65284c48cd864bb6b57dbe5ca0 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a201db721bda4dc3a4422dc8fa9bf72e in term 0.
I20240502 07:59:00.249369 9644 raft_consensus.cc:2454] T bc896a65284c48cd864bb6b57dbe5ca0 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a201db721bda4dc3a4422dc8fa9bf72e in term 0.
I20240502 07:59:00.249646 9451 leader_election.cc:302] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1472c64688914149a318c7b8850a3df3, a201db721bda4dc3a4422dc8fa9bf72e; no voters:
I20240502 07:59:00.249811 10511 raft_consensus.cc:2781] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:00.249871 10511 raft_consensus.cc:482] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:00.249905 10511 raft_consensus.cc:3037] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:00.250836 10511 raft_consensus.cc:504] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:59:00.251150 10511 leader_election.cc:288] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e [CANDIDATE]: Term 1 election: Requested vote from peers 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667), 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191)
I20240502 07:59:00.251435 9777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc896a65284c48cd864bb6b57dbe5ca0" candidate_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1472c64688914149a318c7b8850a3df3"
I20240502 07:59:00.251435 9644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc896a65284c48cd864bb6b57dbe5ca0" candidate_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8337af1943564c0dad18b45a597b805a"
I20240502 07:59:00.251544 9644 raft_consensus.cc:3037] T bc896a65284c48cd864bb6b57dbe5ca0 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:00.251544 9777 raft_consensus.cc:3037] T bc896a65284c48cd864bb6b57dbe5ca0 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:00.252404 9777 raft_consensus.cc:2454] T bc896a65284c48cd864bb6b57dbe5ca0 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a201db721bda4dc3a4422dc8fa9bf72e in term 1.
I20240502 07:59:00.252405 9644 raft_consensus.cc:2454] T bc896a65284c48cd864bb6b57dbe5ca0 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a201db721bda4dc3a4422dc8fa9bf72e in term 1.
I20240502 07:59:00.252665 9449 leader_election.cc:302] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8337af1943564c0dad18b45a597b805a, a201db721bda4dc3a4422dc8fa9bf72e; no voters:
I20240502 07:59:00.252843 10511 raft_consensus.cc:2781] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:00.252956 10511 raft_consensus.cc:686] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Becoming Leader. State: Replica: a201db721bda4dc3a4422dc8fa9bf72e, State: Running, Role: LEADER
I20240502 07:59:00.253087 10511 consensus_queue.cc:237] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:59:00.254678 9372 catalog_manager.cc:5484] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e reported cstate change: term changed from 0 to 1, leader changed from <none> to a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65). New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: HEALTHY } } }
I20240502 07:59:00.303074 10782 raft_consensus.cc:482] T fd251de859e049c38a632a17b2fbdeae P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:00.303171 10782 raft_consensus.cc:504] T fd251de859e049c38a632a17b2fbdeae P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:59:00.303478 10782 leader_election.cc:288] T fd251de859e049c38a632a17b2fbdeae P 6cc5b9962ed24441bcba0c07b9f5438e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667), a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551)
I20240502 07:59:00.303763 9644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd251de859e049c38a632a17b2fbdeae" candidate_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8337af1943564c0dad18b45a597b805a" is_pre_election: true
I20240502 07:59:00.303786 9514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd251de859e049c38a632a17b2fbdeae" candidate_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" is_pre_election: true
I20240502 07:59:00.303877 9644 raft_consensus.cc:2454] T fd251de859e049c38a632a17b2fbdeae P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cc5b9962ed24441bcba0c07b9f5438e in term 0.
I20240502 07:59:00.303895 9514 raft_consensus.cc:2454] T fd251de859e049c38a632a17b2fbdeae P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cc5b9962ed24441bcba0c07b9f5438e in term 0.
I20240502 07:59:00.304108 9842 leader_election.cc:302] T fd251de859e049c38a632a17b2fbdeae P 6cc5b9962ed24441bcba0c07b9f5438e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cc5b9962ed24441bcba0c07b9f5438e, 8337af1943564c0dad18b45a597b805a; no voters:
I20240502 07:59:00.304270 10782 raft_consensus.cc:2781] T fd251de859e049c38a632a17b2fbdeae P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:00.304328 10782 raft_consensus.cc:482] T fd251de859e049c38a632a17b2fbdeae P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:00.304365 10782 raft_consensus.cc:3037] T fd251de859e049c38a632a17b2fbdeae P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:00.305091 10782 raft_consensus.cc:504] T fd251de859e049c38a632a17b2fbdeae P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:59:00.305423 10782 leader_election.cc:288] T fd251de859e049c38a632a17b2fbdeae P 6cc5b9962ed24441bcba0c07b9f5438e [CANDIDATE]: Term 1 election: Requested vote from peers 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667), a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551)
I20240502 07:59:00.305692 9514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd251de859e049c38a632a17b2fbdeae" candidate_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a201db721bda4dc3a4422dc8fa9bf72e"
I20240502 07:59:00.305707 9644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd251de859e049c38a632a17b2fbdeae" candidate_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8337af1943564c0dad18b45a597b805a"
I20240502 07:59:00.305786 9514 raft_consensus.cc:3037] T fd251de859e049c38a632a17b2fbdeae P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:00.305806 9644 raft_consensus.cc:3037] T fd251de859e049c38a632a17b2fbdeae P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:00.306447 9644 raft_consensus.cc:2454] T fd251de859e049c38a632a17b2fbdeae P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cc5b9962ed24441bcba0c07b9f5438e in term 1.
I20240502 07:59:00.306454 9514 raft_consensus.cc:2454] T fd251de859e049c38a632a17b2fbdeae P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cc5b9962ed24441bcba0c07b9f5438e in term 1.
I20240502 07:59:00.306704 9842 leader_election.cc:302] T fd251de859e049c38a632a17b2fbdeae P 6cc5b9962ed24441bcba0c07b9f5438e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cc5b9962ed24441bcba0c07b9f5438e, 8337af1943564c0dad18b45a597b805a; no voters:
I20240502 07:59:00.306874 10782 raft_consensus.cc:2781] T fd251de859e049c38a632a17b2fbdeae P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:00.307065 10782 raft_consensus.cc:686] T fd251de859e049c38a632a17b2fbdeae P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 LEADER]: Becoming Leader. State: Replica: 6cc5b9962ed24441bcba0c07b9f5438e, State: Running, Role: LEADER
I20240502 07:59:00.307201 10782 consensus_queue.cc:237] T fd251de859e049c38a632a17b2fbdeae P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } }
I20240502 07:59:00.308704 9372 catalog_manager.cc:5484] T fd251de859e049c38a632a17b2fbdeae P 6cc5b9962ed24441bcba0c07b9f5438e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68). New cstate: current_term: 1 leader_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: UNKNOWN } } }
W20240502 07:59:00.442523 11067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:00.442773 11067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:00.445904 11067 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_8'
I20240502 07:59:00.533739 9777 raft_consensus.cc:1261] T bc896a65284c48cd864bb6b57dbe5ca0 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:00.533787 9644 raft_consensus.cc:1261] T bc896a65284c48cd864bb6b57dbe5ca0 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:00.534225 10511 consensus_queue.cc:1035] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:00.534862 9514 raft_consensus.cc:1261] T fd251de859e049c38a632a17b2fbdeae P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:00.534978 9644 raft_consensus.cc:1261] T fd251de859e049c38a632a17b2fbdeae P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:00.535223 10930 consensus_queue.cc:1035] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:00.535524 10782 consensus_queue.cc:1035] T fd251de859e049c38a632a17b2fbdeae P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:00.543074 10782 consensus_queue.cc:1035] T fd251de859e049c38a632a17b2fbdeae P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:00.552405 9514 raft_consensus.cc:1261] T 0e4efe560ef74763b80b70665c11d7f1 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:00.553752 10858 consensus_queue.cc:1035] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:00.556515 9906 raft_consensus.cc:1261] T 0e4efe560ef74763b80b70665c11d7f1 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:00.560355 11062 consensus_queue.cc:1035] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 126.31 ms
time per row: 0.0631551 ms
I20240502 07:59:00.621130 9369 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47034:
name: "rebalance_test_table_9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:59:00.626549 9625 tablet_service.cc:1467] Processing CreateTablet for tablet 8ceb129e9cbe4c469e7dfba419719556 (DEFAULT_TABLE table=rebalance_test_table_9 [id=f0ea2346b9b9482e8343d199275d5819]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:00.626710 9886 tablet_service.cc:1467] Processing CreateTablet for tablet 8ceb129e9cbe4c469e7dfba419719556 (DEFAULT_TABLE table=rebalance_test_table_9 [id=f0ea2346b9b9482e8343d199275d5819]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:00.626894 9625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ceb129e9cbe4c469e7dfba419719556. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:00.626845 9757 tablet_service.cc:1467] Processing CreateTablet for tablet 8ceb129e9cbe4c469e7dfba419719556 (DEFAULT_TABLE table=rebalance_test_table_9 [id=f0ea2346b9b9482e8343d199275d5819]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:00.626996 9886 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ceb129e9cbe4c469e7dfba419719556. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:00.627103 9757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ceb129e9cbe4c469e7dfba419719556. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:00.627427 9888 tablet_service.cc:1467] Processing CreateTablet for tablet 70d4db3c3291490db58678522d356688 (DEFAULT_TABLE table=rebalance_test_table_9 [id=f0ea2346b9b9482e8343d199275d5819]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:00.627768 9887 tablet_service.cc:1467] Processing CreateTablet for tablet 0fab37c27e184a2cacb08f05457fa6ae (DEFAULT_TABLE table=rebalance_test_table_9 [id=f0ea2346b9b9482e8343d199275d5819]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:00.627972 9887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fab37c27e184a2cacb08f05457fa6ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:00.628609 9626 tablet_service.cc:1467] Processing CreateTablet for tablet 70d4db3c3291490db58678522d356688 (DEFAULT_TABLE table=rebalance_test_table_9 [id=f0ea2346b9b9482e8343d199275d5819]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:00.628811 9626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70d4db3c3291490db58678522d356688. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:00.629359 11117 tablet_bootstrap.cc:492] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a: Bootstrap starting.
I20240502 07:59:00.629568 9624 tablet_service.cc:1467] Processing CreateTablet for tablet 0fab37c27e184a2cacb08f05457fa6ae (DEFAULT_TABLE table=rebalance_test_table_9 [id=f0ea2346b9b9482e8343d199275d5819]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:00.629776 9624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fab37c27e184a2cacb08f05457fa6ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:00.630203 11117 tablet_bootstrap.cc:654] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:00.630455 9888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70d4db3c3291490db58678522d356688. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:00.631630 11118 tablet_bootstrap.cc:492] T 8ceb129e9cbe4c469e7dfba419719556 P 1472c64688914149a318c7b8850a3df3: Bootstrap starting.
I20240502 07:59:00.632328 11120 tablet_bootstrap.cc:492] T 70d4db3c3291490db58678522d356688 P 6cc5b9962ed24441bcba0c07b9f5438e: Bootstrap starting.
I20240502 07:59:00.632365 11118 tablet_bootstrap.cc:654] T 8ceb129e9cbe4c469e7dfba419719556 P 1472c64688914149a318c7b8850a3df3: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:00.633255 11120 tablet_bootstrap.cc:654] T 70d4db3c3291490db58678522d356688 P 6cc5b9962ed24441bcba0c07b9f5438e: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:00.633562 9757 tablet_service.cc:1467] Processing CreateTablet for tablet 70d4db3c3291490db58678522d356688 (DEFAULT_TABLE table=rebalance_test_table_9 [id=f0ea2346b9b9482e8343d199275d5819]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:00.633797 9757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70d4db3c3291490db58678522d356688. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:00.634207 11120 tablet_bootstrap.cc:492] T 70d4db3c3291490db58678522d356688 P 6cc5b9962ed24441bcba0c07b9f5438e: No bootstrap required, opened a new log
I20240502 07:59:00.634294 11120 ts_tablet_manager.cc:1367] T 70d4db3c3291490db58678522d356688 P 6cc5b9962ed24441bcba0c07b9f5438e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:00.634646 11120 raft_consensus.cc:348] T 70d4db3c3291490db58678522d356688 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:59:00.634737 11120 raft_consensus.cc:374] T 70d4db3c3291490db58678522d356688 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:00.634764 11120 raft_consensus.cc:729] T 70d4db3c3291490db58678522d356688 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cc5b9962ed24441bcba0c07b9f5438e, State: Initialized, Role: FOLLOWER
I20240502 07:59:00.634893 11120 consensus_queue.cc:260] T 70d4db3c3291490db58678522d356688 P 6cc5b9962ed24441bcba0c07b9f5438e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:59:00.635169 11120 ts_tablet_manager.cc:1398] T 70d4db3c3291490db58678522d356688 P 6cc5b9962ed24441bcba0c07b9f5438e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:00.635339 11120 tablet_bootstrap.cc:492] T 0fab37c27e184a2cacb08f05457fa6ae P 6cc5b9962ed24441bcba0c07b9f5438e: Bootstrap starting.
I20240502 07:59:00.635743 9493 tablet_service.cc:1467] Processing CreateTablet for tablet 0fab37c27e184a2cacb08f05457fa6ae (DEFAULT_TABLE table=rebalance_test_table_9 [id=f0ea2346b9b9482e8343d199275d5819]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:00.636009 9493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fab37c27e184a2cacb08f05457fa6ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:00.636246 11120 tablet_bootstrap.cc:654] T 0fab37c27e184a2cacb08f05457fa6ae P 6cc5b9962ed24441bcba0c07b9f5438e: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:00.637137 11120 tablet_bootstrap.cc:492] T 0fab37c27e184a2cacb08f05457fa6ae P 6cc5b9962ed24441bcba0c07b9f5438e: No bootstrap required, opened a new log
I20240502 07:59:00.637212 11120 ts_tablet_manager.cc:1367] T 0fab37c27e184a2cacb08f05457fa6ae P 6cc5b9962ed24441bcba0c07b9f5438e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:00.637552 11120 raft_consensus.cc:348] T 0fab37c27e184a2cacb08f05457fa6ae P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:59:00.637636 11120 raft_consensus.cc:374] T 0fab37c27e184a2cacb08f05457fa6ae P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:00.637663 11120 raft_consensus.cc:729] T 0fab37c27e184a2cacb08f05457fa6ae P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cc5b9962ed24441bcba0c07b9f5438e, State: Initialized, Role: FOLLOWER
I20240502 07:59:00.637854 11124 tablet_bootstrap.cc:492] T 0fab37c27e184a2cacb08f05457fa6ae P a201db721bda4dc3a4422dc8fa9bf72e: Bootstrap starting.
I20240502 07:59:00.637817 11120 consensus_queue.cc:260] T 0fab37c27e184a2cacb08f05457fa6ae P 6cc5b9962ed24441bcba0c07b9f5438e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:59:00.638834 11124 tablet_bootstrap.cc:654] T 0fab37c27e184a2cacb08f05457fa6ae P a201db721bda4dc3a4422dc8fa9bf72e: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:00.639227 11120 ts_tablet_manager.cc:1398] T 0fab37c27e184a2cacb08f05457fa6ae P 6cc5b9962ed24441bcba0c07b9f5438e: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:00.639381 11120 tablet_bootstrap.cc:492] T 8ceb129e9cbe4c469e7dfba419719556 P 6cc5b9962ed24441bcba0c07b9f5438e: Bootstrap starting.
I20240502 07:59:00.639519 11117 tablet_bootstrap.cc:492] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a: No bootstrap required, opened a new log
I20240502 07:59:00.639626 11117 ts_tablet_manager.cc:1367] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 07:59:00.640054 11117 raft_consensus.cc:348] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:59:00.640161 11117 raft_consensus.cc:374] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:00.640170 11120 tablet_bootstrap.cc:654] T 8ceb129e9cbe4c469e7dfba419719556 P 6cc5b9962ed24441bcba0c07b9f5438e: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:00.640197 11117 raft_consensus.cc:729] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8337af1943564c0dad18b45a597b805a, State: Initialized, Role: FOLLOWER
I20240502 07:59:00.640336 11117 consensus_queue.cc:260] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:59:00.640471 11124 tablet_bootstrap.cc:492] T 0fab37c27e184a2cacb08f05457fa6ae P a201db721bda4dc3a4422dc8fa9bf72e: No bootstrap required, opened a new log
I20240502 07:59:00.640564 11124 ts_tablet_manager.cc:1367] T 0fab37c27e184a2cacb08f05457fa6ae P a201db721bda4dc3a4422dc8fa9bf72e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:00.640602 11117 ts_tablet_manager.cc:1398] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:00.640779 11117 tablet_bootstrap.cc:492] T 70d4db3c3291490db58678522d356688 P 8337af1943564c0dad18b45a597b805a: Bootstrap starting.
I20240502 07:59:00.640990 11124 raft_consensus.cc:348] T 0fab37c27e184a2cacb08f05457fa6ae P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:59:00.641065 11118 tablet_bootstrap.cc:492] T 8ceb129e9cbe4c469e7dfba419719556 P 1472c64688914149a318c7b8850a3df3: No bootstrap required, opened a new log
I20240502 07:59:00.641079 11124 raft_consensus.cc:374] T 0fab37c27e184a2cacb08f05457fa6ae P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:00.641106 11124 raft_consensus.cc:729] T 0fab37c27e184a2cacb08f05457fa6ae P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a201db721bda4dc3a4422dc8fa9bf72e, State: Initialized, Role: FOLLOWER
I20240502 07:59:00.641142 11118 ts_tablet_manager.cc:1367] T 8ceb129e9cbe4c469e7dfba419719556 P 1472c64688914149a318c7b8850a3df3: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20240502 07:59:00.641213 11124 consensus_queue.cc:260] T 0fab37c27e184a2cacb08f05457fa6ae P a201db721bda4dc3a4422dc8fa9bf72e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:59:00.641480 11124 ts_tablet_manager.cc:1398] T 0fab37c27e184a2cacb08f05457fa6ae P a201db721bda4dc3a4422dc8fa9bf72e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:00.641569 11117 tablet_bootstrap.cc:654] T 70d4db3c3291490db58678522d356688 P 8337af1943564c0dad18b45a597b805a: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:00.642544 11117 tablet_bootstrap.cc:492] T 70d4db3c3291490db58678522d356688 P 8337af1943564c0dad18b45a597b805a: No bootstrap required, opened a new log
I20240502 07:59:00.642621 11117 ts_tablet_manager.cc:1367] T 70d4db3c3291490db58678522d356688 P 8337af1943564c0dad18b45a597b805a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:00.643015 11117 raft_consensus.cc:348] T 70d4db3c3291490db58678522d356688 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:59:00.643150 11117 raft_consensus.cc:374] T 70d4db3c3291490db58678522d356688 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:00.643185 11117 raft_consensus.cc:729] T 70d4db3c3291490db58678522d356688 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8337af1943564c0dad18b45a597b805a, State: Initialized, Role: FOLLOWER
I20240502 07:59:00.643309 11117 consensus_queue.cc:260] T 70d4db3c3291490db58678522d356688 P 8337af1943564c0dad18b45a597b805a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:59:00.644052 11117 ts_tablet_manager.cc:1398] T 70d4db3c3291490db58678522d356688 P 8337af1943564c0dad18b45a597b805a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:00.644212 11117 tablet_bootstrap.cc:492] T 0fab37c27e184a2cacb08f05457fa6ae P 8337af1943564c0dad18b45a597b805a: Bootstrap starting.
I20240502 07:59:00.644212 11120 tablet_bootstrap.cc:492] T 8ceb129e9cbe4c469e7dfba419719556 P 6cc5b9962ed24441bcba0c07b9f5438e: No bootstrap required, opened a new log
I20240502 07:59:00.644287 11120 ts_tablet_manager.cc:1367] T 8ceb129e9cbe4c469e7dfba419719556 P 6cc5b9962ed24441bcba0c07b9f5438e: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:59:00.644810 11120 raft_consensus.cc:348] T 8ceb129e9cbe4c469e7dfba419719556 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:59:00.644906 11120 raft_consensus.cc:374] T 8ceb129e9cbe4c469e7dfba419719556 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:00.644937 11120 raft_consensus.cc:729] T 8ceb129e9cbe4c469e7dfba419719556 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cc5b9962ed24441bcba0c07b9f5438e, State: Initialized, Role: FOLLOWER
I20240502 07:59:00.645049 11117 tablet_bootstrap.cc:654] T 0fab37c27e184a2cacb08f05457fa6ae P 8337af1943564c0dad18b45a597b805a: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:00.645085 11120 consensus_queue.cc:260] T 8ceb129e9cbe4c469e7dfba419719556 P 6cc5b9962ed24441bcba0c07b9f5438e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:59:00.645445 11120 ts_tablet_manager.cc:1398] T 8ceb129e9cbe4c469e7dfba419719556 P 6cc5b9962ed24441bcba0c07b9f5438e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:00.645875 11118 raft_consensus.cc:348] T 8ceb129e9cbe4c469e7dfba419719556 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:59:00.645974 11118 raft_consensus.cc:374] T 8ceb129e9cbe4c469e7dfba419719556 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:00.646005 11118 raft_consensus.cc:729] T 8ceb129e9cbe4c469e7dfba419719556 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1472c64688914149a318c7b8850a3df3, State: Initialized, Role: FOLLOWER
I20240502 07:59:00.646134 11118 consensus_queue.cc:260] T 8ceb129e9cbe4c469e7dfba419719556 P 1472c64688914149a318c7b8850a3df3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:59:00.646498 11118 ts_tablet_manager.cc:1398] T 8ceb129e9cbe4c469e7dfba419719556 P 1472c64688914149a318c7b8850a3df3: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20240502 07:59:00.646639 11118 tablet_bootstrap.cc:492] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3: Bootstrap starting.
I20240502 07:59:00.647523 11118 tablet_bootstrap.cc:654] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:00.648828 11118 tablet_bootstrap.cc:492] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3: No bootstrap required, opened a new log
I20240502 07:59:00.648859 11117 tablet_bootstrap.cc:492] T 0fab37c27e184a2cacb08f05457fa6ae P 8337af1943564c0dad18b45a597b805a: No bootstrap required, opened a new log
I20240502 07:59:00.648908 11118 ts_tablet_manager.cc:1367] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:00.648931 11117 ts_tablet_manager.cc:1367] T 0fab37c27e184a2cacb08f05457fa6ae P 8337af1943564c0dad18b45a597b805a: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 07:59:00.649266 11117 raft_consensus.cc:348] T 0fab37c27e184a2cacb08f05457fa6ae P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:59:00.649355 11117 raft_consensus.cc:374] T 0fab37c27e184a2cacb08f05457fa6ae P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:00.649298 11118 raft_consensus.cc:348] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:59:00.649384 11117 raft_consensus.cc:729] T 0fab37c27e184a2cacb08f05457fa6ae P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8337af1943564c0dad18b45a597b805a, State: Initialized, Role: FOLLOWER
I20240502 07:59:00.649395 11118 raft_consensus.cc:374] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:00.649425 11118 raft_consensus.cc:729] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1472c64688914149a318c7b8850a3df3, State: Initialized, Role: FOLLOWER
I20240502 07:59:00.649520 11117 consensus_queue.cc:260] T 0fab37c27e184a2cacb08f05457fa6ae P 8337af1943564c0dad18b45a597b805a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:59:00.649546 11118 consensus_queue.cc:260] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:59:00.649791 11118 ts_tablet_manager.cc:1398] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:00.649884 11117 ts_tablet_manager.cc:1398] T 0fab37c27e184a2cacb08f05457fa6ae P 8337af1943564c0dad18b45a597b805a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:00.731552 11065 raft_consensus.cc:482] T 0fab37c27e184a2cacb08f05457fa6ae P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:00.731674 11065 raft_consensus.cc:504] T 0fab37c27e184a2cacb08f05457fa6ae P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:59:00.732061 11065 leader_election.cc:288] T 0fab37c27e184a2cacb08f05457fa6ae P 6cc5b9962ed24441bcba0c07b9f5438e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551), 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667)
I20240502 07:59:00.732419 9515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fab37c27e184a2cacb08f05457fa6ae" candidate_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" is_pre_election: true
I20240502 07:59:00.732411 9644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fab37c27e184a2cacb08f05457fa6ae" candidate_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8337af1943564c0dad18b45a597b805a" is_pre_election: true
I20240502 07:59:00.732553 9515 raft_consensus.cc:2454] T 0fab37c27e184a2cacb08f05457fa6ae P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cc5b9962ed24441bcba0c07b9f5438e in term 0.
I20240502 07:59:00.732553 9644 raft_consensus.cc:2454] T 0fab37c27e184a2cacb08f05457fa6ae P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cc5b9962ed24441bcba0c07b9f5438e in term 0.
I20240502 07:59:00.732825 9842 leader_election.cc:302] T 0fab37c27e184a2cacb08f05457fa6ae P 6cc5b9962ed24441bcba0c07b9f5438e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cc5b9962ed24441bcba0c07b9f5438e, 8337af1943564c0dad18b45a597b805a; no voters:
I20240502 07:59:00.732991 11065 raft_consensus.cc:2781] T 0fab37c27e184a2cacb08f05457fa6ae P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:00.733047 11065 raft_consensus.cc:482] T 0fab37c27e184a2cacb08f05457fa6ae P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:00.733073 11065 raft_consensus.cc:3037] T 0fab37c27e184a2cacb08f05457fa6ae P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:00.733920 11065 raft_consensus.cc:504] T 0fab37c27e184a2cacb08f05457fa6ae P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:59:00.734226 11065 leader_election.cc:288] T 0fab37c27e184a2cacb08f05457fa6ae P 6cc5b9962ed24441bcba0c07b9f5438e [CANDIDATE]: Term 1 election: Requested vote from peers a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551), 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667)
I20240502 07:59:00.734516 9515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fab37c27e184a2cacb08f05457fa6ae" candidate_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a201db721bda4dc3a4422dc8fa9bf72e"
I20240502 07:59:00.734541 9644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fab37c27e184a2cacb08f05457fa6ae" candidate_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8337af1943564c0dad18b45a597b805a"
I20240502 07:59:00.734618 9515 raft_consensus.cc:3037] T 0fab37c27e184a2cacb08f05457fa6ae P a201db721bda4dc3a4422dc8fa9bf72e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:00.734632 9644 raft_consensus.cc:3037] T 0fab37c27e184a2cacb08f05457fa6ae P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:00.735423 9644 raft_consensus.cc:2454] T 0fab37c27e184a2cacb08f05457fa6ae P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cc5b9962ed24441bcba0c07b9f5438e in term 1.
I20240502 07:59:00.735462 9515 raft_consensus.cc:2454] T 0fab37c27e184a2cacb08f05457fa6ae P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cc5b9962ed24441bcba0c07b9f5438e in term 1.
I20240502 07:59:00.735684 9842 leader_election.cc:302] T 0fab37c27e184a2cacb08f05457fa6ae P 6cc5b9962ed24441bcba0c07b9f5438e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cc5b9962ed24441bcba0c07b9f5438e, 8337af1943564c0dad18b45a597b805a; no voters:
I20240502 07:59:00.735842 11065 raft_consensus.cc:2781] T 0fab37c27e184a2cacb08f05457fa6ae P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:00.735913 11065 raft_consensus.cc:686] T 0fab37c27e184a2cacb08f05457fa6ae P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 LEADER]: Becoming Leader. State: Replica: 6cc5b9962ed24441bcba0c07b9f5438e, State: Running, Role: LEADER
I20240502 07:59:00.736045 11065 consensus_queue.cc:237] T 0fab37c27e184a2cacb08f05457fa6ae P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:59:00.738034 9365 catalog_manager.cc:5484] T 0fab37c27e184a2cacb08f05457fa6ae P 6cc5b9962ed24441bcba0c07b9f5438e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68). New cstate: current_term: 1 leader_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:00.758071 10891 raft_consensus.cc:482] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:00.758180 10891 raft_consensus.cc:504] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:59:00.758549 10891 leader_election.cc:288] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379), 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191)
I20240502 07:59:00.758845 9907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ceb129e9cbe4c469e7dfba419719556" candidate_uuid: "8337af1943564c0dad18b45a597b805a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" is_pre_election: true
I20240502 07:59:00.758899 9777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ceb129e9cbe4c469e7dfba419719556" candidate_uuid: "8337af1943564c0dad18b45a597b805a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1472c64688914149a318c7b8850a3df3" is_pre_election: true
I20240502 07:59:00.758982 9907 raft_consensus.cc:2454] T 8ceb129e9cbe4c469e7dfba419719556 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8337af1943564c0dad18b45a597b805a in term 0.
I20240502 07:59:00.759020 9777 raft_consensus.cc:2454] T 8ceb129e9cbe4c469e7dfba419719556 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8337af1943564c0dad18b45a597b805a in term 0.
I20240502 07:59:00.759295 9580 leader_election.cc:302] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cc5b9962ed24441bcba0c07b9f5438e, 8337af1943564c0dad18b45a597b805a; no voters:
I20240502 07:59:00.759497 10891 raft_consensus.cc:2781] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:00.759570 10891 raft_consensus.cc:482] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:00.759608 10891 raft_consensus.cc:3037] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:00.762221 10891 raft_consensus.cc:504] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:59:00.762514 10891 leader_election.cc:288] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a [CANDIDATE]: Term 1 election: Requested vote from peers 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379), 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191)
I20240502 07:59:00.762794 9907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ceb129e9cbe4c469e7dfba419719556" candidate_uuid: "8337af1943564c0dad18b45a597b805a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cc5b9962ed24441bcba0c07b9f5438e"
I20240502 07:59:00.762852 9777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ceb129e9cbe4c469e7dfba419719556" candidate_uuid: "8337af1943564c0dad18b45a597b805a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1472c64688914149a318c7b8850a3df3"
I20240502 07:59:00.762902 9907 raft_consensus.cc:3037] T 8ceb129e9cbe4c469e7dfba419719556 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:00.762965 9777 raft_consensus.cc:3037] T 8ceb129e9cbe4c469e7dfba419719556 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:00.763836 9907 raft_consensus.cc:2454] T 8ceb129e9cbe4c469e7dfba419719556 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8337af1943564c0dad18b45a597b805a in term 1.
I20240502 07:59:00.763856 9777 raft_consensus.cc:2454] T 8ceb129e9cbe4c469e7dfba419719556 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8337af1943564c0dad18b45a597b805a in term 1.
I20240502 07:59:00.764106 9580 leader_election.cc:302] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cc5b9962ed24441bcba0c07b9f5438e, 8337af1943564c0dad18b45a597b805a; no voters:
I20240502 07:59:00.764312 10891 raft_consensus.cc:2781] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:00.764392 10891 raft_consensus.cc:686] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Becoming Leader. State: Replica: 8337af1943564c0dad18b45a597b805a, State: Running, Role: LEADER
I20240502 07:59:00.764514 10891 consensus_queue.cc:237] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } }
I20240502 07:59:00.767225 9365 catalog_manager.cc:5484] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a reported cstate change: term changed from 0 to 1, leader changed from <none> to 8337af1943564c0dad18b45a597b805a (127.3.139.66). New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:00.801147 10858 raft_consensus.cc:482] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:00.801267 10858 raft_consensus.cc:504] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:59:00.801630 10858 leader_election.cc:288] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379), 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667)
I20240502 07:59:00.802016 9907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70d4db3c3291490db58678522d356688" candidate_uuid: "1472c64688914149a318c7b8850a3df3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" is_pre_election: true
I20240502 07:59:00.802031 9644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70d4db3c3291490db58678522d356688" candidate_uuid: "1472c64688914149a318c7b8850a3df3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8337af1943564c0dad18b45a597b805a" is_pre_election: true
I20240502 07:59:00.802145 9907 raft_consensus.cc:2454] T 70d4db3c3291490db58678522d356688 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1472c64688914149a318c7b8850a3df3 in term 0.
I20240502 07:59:00.802153 9644 raft_consensus.cc:2454] T 70d4db3c3291490db58678522d356688 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1472c64688914149a318c7b8850a3df3 in term 0.
I20240502 07:59:00.802428 9711 leader_election.cc:302] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1472c64688914149a318c7b8850a3df3, 6cc5b9962ed24441bcba0c07b9f5438e; no voters:
I20240502 07:59:00.802589 10858 raft_consensus.cc:2781] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:00.802641 10858 raft_consensus.cc:482] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:00.802681 10858 raft_consensus.cc:3037] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:00.803467 10858 raft_consensus.cc:504] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:59:00.803755 10858 leader_election.cc:288] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 [CANDIDATE]: Term 1 election: Requested vote from peers 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379), 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667)
I20240502 07:59:00.804075 9907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70d4db3c3291490db58678522d356688" candidate_uuid: "1472c64688914149a318c7b8850a3df3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cc5b9962ed24441bcba0c07b9f5438e"
I20240502 07:59:00.804132 9644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70d4db3c3291490db58678522d356688" candidate_uuid: "1472c64688914149a318c7b8850a3df3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8337af1943564c0dad18b45a597b805a"
I20240502 07:59:00.804201 9907 raft_consensus.cc:3037] T 70d4db3c3291490db58678522d356688 P 6cc5b9962ed24441bcba0c07b9f5438e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:00.804216 9644 raft_consensus.cc:3037] T 70d4db3c3291490db58678522d356688 P 8337af1943564c0dad18b45a597b805a [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:00.805091 9644 raft_consensus.cc:2454] T 70d4db3c3291490db58678522d356688 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1472c64688914149a318c7b8850a3df3 in term 1.
I20240502 07:59:00.805091 9907 raft_consensus.cc:2454] T 70d4db3c3291490db58678522d356688 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1472c64688914149a318c7b8850a3df3 in term 1.
I20240502 07:59:00.805375 9711 leader_election.cc:302] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1472c64688914149a318c7b8850a3df3, 6cc5b9962ed24441bcba0c07b9f5438e; no voters:
I20240502 07:59:00.805563 10858 raft_consensus.cc:2781] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:00.805660 10858 raft_consensus.cc:686] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Becoming Leader. State: Replica: 1472c64688914149a318c7b8850a3df3, State: Running, Role: LEADER
I20240502 07:59:00.805811 10858 consensus_queue.cc:237] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } }
I20240502 07:59:00.807529 9365 catalog_manager.cc:5484] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1472c64688914149a318c7b8850a3df3 (127.3.139.67). New cstate: current_term: 1 leader_uuid: "1472c64688914149a318c7b8850a3df3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: UNKNOWN } } }
W20240502 07:59:00.924595 11133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:00.924793 11133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:00.927693 11133 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_9'
I20240502 07:59:01.016578 9644 raft_consensus.cc:1261] T 70d4db3c3291490db58678522d356688 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:01.016578 9907 raft_consensus.cc:1261] T 70d4db3c3291490db58678522d356688 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:01.017022 11062 consensus_queue.cc:1035] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:01.017321 11062 consensus_queue.cc:1035] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:01.017905 9644 raft_consensus.cc:1261] T 0fab37c27e184a2cacb08f05457fa6ae P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:01.018395 9515 raft_consensus.cc:1261] T 0fab37c27e184a2cacb08f05457fa6ae P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:01.019076 11065 consensus_queue.cc:1035] T 0fab37c27e184a2cacb08f05457fa6ae P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:01.019356 11065 consensus_queue.cc:1035] T 0fab37c27e184a2cacb08f05457fa6ae P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:01.019121 9907 raft_consensus.cc:1261] T 8ceb129e9cbe4c469e7dfba419719556 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:01.022168 9777 raft_consensus.cc:1261] T 8ceb129e9cbe4c469e7dfba419719556 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:01.022616 11021 consensus_queue.cc:1035] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:01.024757 10891 consensus_queue.cc:1035] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 110.76 ms
time per row: 0.05538 ms
I20240502 07:59:01.078745 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.69:45345
--local_ip_for_outbound_sockets=127.3.139.69
--tserver_master_addrs=127.3.139.126:46479
--webserver_port=37901
--webserver_interface=127.3.139.69
--builtin_ntp_servers=127.3.139.84:39259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
W20240502 07:59:01.185294 11178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:01.185480 11178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:59:01.185528 11178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:01.188438 11178 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:59:01.188493 11178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:59:01.188571 11178 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.69
I20240502 07:59:01.191952 11178 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:39259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.69:45345
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.3.139.69
--webserver_port=37901
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.3.139.126:46479
--never_fsync=true
--heap_profile_path=/tmp/kudu.11178
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.69
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:59:01.192705 11178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:59:01.193820 11178 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:59:01.201332 11183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:01.201704 11184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:01.202109 11186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:01.202502 11178 server_base.cc:1034] running on GCE node
I20240502 07:59:01.202836 11178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:59:01.203423 11178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:59:01.204582 11178 hybrid_clock.cc:648] HybridClock initialized: now 1714636741204557 us; error 41 us; skew 500 ppm
I20240502 07:59:01.212162 11178 webserver.cc:460] Webserver started at http://127.3.139.69:37901/ using document root <none> and password file <none>
I20240502 07:59:01.212692 11178 fs_manager.cc:359] Metadata directory not provided
I20240502 07:59:01.212759 11178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:59:01.216228 11178 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240502 07:59:01.218199 11192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:01.219226 11178 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240502 07:59:01.219401 11178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "51e82953ed2247fa9ff24638eab3508a"
format_stamp: "Formatted at 2024-05-02 07:58:51 on dist-test-slave-twcr"
I20240502 07:59:01.219810 11178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:59:01.249413 11178 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:59:01.250020 11178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:59:01.250172 11178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:59:01.250797 11178 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:59:01.251792 11178 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:59:01.251839 11178 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:01.251878 11178 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:59:01.251899 11178 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:01.260278 11178 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.69:45345
I20240502 07:59:01.260315 11305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.69:45345 every 8 connection(s)
I20240502 07:59:01.261238 11178 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240502 07:59:01.264071 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 11178
I20240502 07:59:01.264513 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.70:45329
--local_ip_for_outbound_sockets=127.3.139.70
--tserver_master_addrs=127.3.139.126:46479
--webserver_port=37095
--webserver_interface=127.3.139.70
--builtin_ntp_servers=127.3.139.84:39259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240502 07:59:01.270238 11306 heartbeater.cc:347] Connected to a master server at 127.3.139.126:46479
I20240502 07:59:01.270478 11306 heartbeater.cc:464] Registering TS with master...
I20240502 07:59:01.270984 11306 heartbeater.cc:511] Master 127.3.139.126:46479 requested a full tablet report, sending...
I20240502 07:59:01.272162 9365 ts_manager.cc:194] Re-registered known tserver with Master: 51e82953ed2247fa9ff24638eab3508a (127.3.139.69:45345)
I20240502 07:59:01.272728 9365 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.69:36421
W20240502 07:59:01.375388 11309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:01.375584 11309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:59:01.375635 11309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:01.378525 11309 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:59:01.378582 11309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:59:01.378671 11309 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.70
I20240502 07:59:01.382112 11309 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:39259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.70:45329
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.3.139.70
--webserver_port=37095
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.3.139.126:46479
--never_fsync=true
--heap_profile_path=/tmp/kudu.11309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.70
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:59:01.382853 11309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:59:01.383862 11309 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:59:01.391486 11317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:01.391806 11314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:01.392168 11315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:01.392585 11309 server_base.cc:1034] running on GCE node
I20240502 07:59:01.392907 11309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:59:01.393383 11309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:59:01.394527 11309 hybrid_clock.cc:648] HybridClock initialized: now 1714636741394516 us; error 22 us; skew 500 ppm
I20240502 07:59:01.396140 11309 webserver.cc:460] Webserver started at http://127.3.139.70:37095/ using document root <none> and password file <none>
I20240502 07:59:01.396621 11309 fs_manager.cc:359] Metadata directory not provided
I20240502 07:59:01.396688 11309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:59:01.399653 11309 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240502 07:59:01.401443 11323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:01.402323 11309 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:01.402410 11309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "8eb9945f989d4be6b51e1531a1365350"
format_stamp: "Formatted at 2024-05-02 07:58:51 on dist-test-slave-twcr"
I20240502 07:59:01.402750 11309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:59:01.416529 11309 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:59:01.416975 11309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:59:01.417101 11309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:59:01.417538 11309 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:59:01.418285 11309 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:59:01.418329 11309 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:01.418367 11309 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:59:01.418394 11309 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:01.426308 11309 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.70:45329
I20240502 07:59:01.426335 11436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.70:45329 every 8 connection(s)
I20240502 07:59:01.427193 11309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240502 07:59:01.429620 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 11309
I20240502 07:59:01.429971 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.71:46577
--local_ip_for_outbound_sockets=127.3.139.71
--tserver_master_addrs=127.3.139.126:46479
--webserver_port=36755
--webserver_interface=127.3.139.71
--builtin_ntp_servers=127.3.139.84:39259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240502 07:59:01.435101 11437 heartbeater.cc:347] Connected to a master server at 127.3.139.126:46479
I20240502 07:59:01.435281 11437 heartbeater.cc:464] Registering TS with master...
I20240502 07:59:01.435707 11437 heartbeater.cc:511] Master 127.3.139.126:46479 requested a full tablet report, sending...
I20240502 07:59:01.436601 9365 ts_manager.cc:194] Re-registered known tserver with Master: 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70:45329)
I20240502 07:59:01.437314 9365 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.70:37953
W20240502 07:59:01.536476 11440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:01.536720 11440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:59:01.536770 11440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:01.539752 11440 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:59:01.539808 11440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:59:01.539876 11440 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.71
I20240502 07:59:01.543617 11440 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:39259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.71:46577
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.3.139.71
--webserver_port=36755
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.3.139.126:46479
--never_fsync=true
--heap_profile_path=/tmp/kudu.11440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.71
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:59:01.544497 11440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:59:01.545589 11440 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:59:01.553697 11445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:01.554023 11448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:01.554447 11446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:01.554895 11440 server_base.cc:1034] running on GCE node
I20240502 07:59:01.555377 11440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:59:01.556013 11440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:59:01.557179 11440 hybrid_clock.cc:648] HybridClock initialized: now 1714636741557121 us; error 71 us; skew 500 ppm
I20240502 07:59:01.559139 11440 webserver.cc:460] Webserver started at http://127.3.139.71:36755/ using document root <none> and password file <none>
I20240502 07:59:01.559736 11440 fs_manager.cc:359] Metadata directory not provided
I20240502 07:59:01.559814 11440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:59:01.563390 11440 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20240502 07:59:01.565327 11454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:01.566344 11440 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:01.566442 11440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "98bef2e94ade4763a66c1ad68259744a"
format_stamp: "Formatted at 2024-05-02 07:58:51 on dist-test-slave-twcr"
I20240502 07:59:01.566823 11440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:59:01.588313 11440 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:59:01.588989 11440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:59:01.589143 11440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:59:01.589689 11440 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:59:01.590632 11440 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:59:01.590679 11440 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:01.590731 11440 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:59:01.590755 11440 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:01.599963 11440 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.71:46577
I20240502 07:59:01.600071 11567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.71:46577 every 8 connection(s)
I20240502 07:59:01.601161 11440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240502 07:59:01.605391 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 11440
I20240502 07:59:01.605770 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.72:36271
--local_ip_for_outbound_sockets=127.3.139.72
--tserver_master_addrs=127.3.139.126:46479
--webserver_port=46209
--webserver_interface=127.3.139.72
--builtin_ntp_servers=127.3.139.84:39259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240502 07:59:01.611096 11568 heartbeater.cc:347] Connected to a master server at 127.3.139.126:46479
I20240502 07:59:01.611322 11568 heartbeater.cc:464] Registering TS with master...
I20240502 07:59:01.611819 11568 heartbeater.cc:511] Master 127.3.139.126:46479 requested a full tablet report, sending...
I20240502 07:59:01.612795 9365 ts_manager.cc:194] Re-registered known tserver with Master: 98bef2e94ade4763a66c1ad68259744a (127.3.139.71:46577)
I20240502 07:59:01.613317 9365 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.71:34187
W20240502 07:59:01.711741 11571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:01.711944 11571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:59:01.712003 11571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:01.715006 11571 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:59:01.715092 11571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:59:01.715183 11571 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.72
I20240502 07:59:01.718745 11571 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:39259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.72:36271
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.3.139.72
--webserver_port=46209
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.3.139.126:46479
--never_fsync=true
--heap_profile_path=/tmp/kudu.11571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.72
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:59:01.719635 11571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:59:01.720700 11571 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:59:01.728658 11578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:01.729027 11579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:01.729357 11581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:01.729758 11571 server_base.cc:1034] running on GCE node
I20240502 07:59:01.730321 11571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:59:01.730942 11571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:59:01.732095 11571 hybrid_clock.cc:648] HybridClock initialized: now 1714636741732066 us; error 39 us; skew 500 ppm
I20240502 07:59:01.734297 11571 webserver.cc:460] Webserver started at http://127.3.139.72:46209/ using document root <none> and password file <none>
I20240502 07:59:01.734925 11571 fs_manager.cc:359] Metadata directory not provided
I20240502 07:59:01.735015 11571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:59:01.738272 11571 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:01.740128 11587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:01.741151 11571 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:01.741251 11571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "a95f1d3af33f4c879fd290ca917d6329"
format_stamp: "Formatted at 2024-05-02 07:58:51 on dist-test-slave-twcr"
I20240502 07:59:01.741640 11571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:59:01.782636 11571 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:59:01.783313 11571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:59:01.783463 11571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:59:01.784036 11571 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:59:01.785039 11571 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:59:01.785094 11571 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:01.785132 11571 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:59:01.785153 11571 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:01.792999 11571 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.72:36271
I20240502 07:59:01.793025 11700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.72:36271 every 8 connection(s)
I20240502 07:59:01.793936 11571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240502 07:59:01.801496 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 11571
I20240502 07:59:01.802007 11701 heartbeater.cc:347] Connected to a master server at 127.3.139.126:46479
I20240502 07:59:01.802210 11701 heartbeater.cc:464] Registering TS with master...
I20240502 07:59:01.802680 11701 heartbeater.cc:511] Master 127.3.139.126:46479 requested a full tablet report, sending...
I20240502 07:59:01.804836 9365 ts_manager.cc:194] Re-registered known tserver with Master: a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72:36271)
I20240502 07:59:01.805415 9365 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.72:33659
W20240502 07:59:01.928010 11705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:01.928251 11705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:01.931198 11705 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20240502 07:59:02.053900 9886 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:02.057406 11502 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:02.059540 11371 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:02.061093 9757 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:02.065075 9493 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:02.086764 11635 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:02.095201 9624 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:02.095726 11240 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:02.096674 9624 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:02.108644 9493 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:02.111410 11635 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:02.111685 11502 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:02.111887 11240 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:02.120160 9886 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:02.128813 9757 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:02.135185 11371 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:02.272661 9757 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:02.275100 11306 heartbeater.cc:503] Master 127.3.139.126:46479 was elected leader, sending a full tablet report...
I20240502 07:59:02.279333 11371 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:02.282125 9624 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:02.283972 11240 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:02.291740 9886 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:02.292809 11502 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:02.313191 9493 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:02.315106 11635 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
W20240502 07:59:02.372251 11816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:02.372509 11816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:02.375636 11816 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20240502 07:59:02.438963 11437 heartbeater.cc:503] Master 127.3.139.126:46479 was elected leader, sending a full tablet report...
I20240502 07:59:02.443539 9624 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:02.444207 9757 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:02.451572 11502 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:02.457283 11240 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:02.459849 9886 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:02.465631 11635 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:02.467795 9493 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:02.468180 11371 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:02.531499 11240 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:02.535014 9886 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:02.540161 9776 consensus_queue.cc:237] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } }
I20240502 07:59:02.541464 11371 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:02.542366 9515 raft_consensus.cc:1261] T 92f37161960e4ee894c75d9034434001 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:02.542776 11062 consensus_queue.cc:1035] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:02.544226 9908 raft_consensus.cc:1261] T 92f37161960e4ee894c75d9034434001 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:02.544611 11914 consensus_queue.cc:1035] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:02.545497 11635 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:02.552969 9757 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
W20240502 07:59:02.557570 9711 consensus_peers.cc:485] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 -> Peer 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70:45329): Couldn't send request to peer 8eb9945f989d4be6b51e1531a1365350. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 92f37161960e4ee894c75d9034434001. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:02.559562 9624 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:02.559823 11502 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:02.569506 9493 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:02.580185 11914 raft_consensus.cc:2932] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } } }
I20240502 07:59:02.580399 9515 raft_consensus.cc:2932] T 92f37161960e4ee894c75d9034434001 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } } }
I20240502 07:59:02.582758 9908 raft_consensus.cc:2932] T 92f37161960e4ee894c75d9034434001 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } } }
I20240502 07:59:02.583319 9369 catalog_manager.cc:5484] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 reported cstate change: config changed from index -1 to 4, NON_VOTER 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) added. New cstate: current_term: 1 leader_uuid: "1472c64688914149a318c7b8850a3df3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:02.614027 9515 consensus_queue.cc:237] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } }
I20240502 07:59:02.615317 11568 heartbeater.cc:503] Master 127.3.139.126:46479 was elected leader, sending a full tablet report...
I20240502 07:59:02.618023 9908 raft_consensus.cc:1261] T a12a084dba23425996e9122c083c1444 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:02.619112 9776 raft_consensus.cc:1261] T a12a084dba23425996e9122c083c1444 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:02.619594 11572 consensus_queue.cc:1035] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:02.621127 11572 consensus_queue.cc:1035] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 07:59:02.622504 9451 consensus_peers.cc:485] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e -> Peer 51e82953ed2247fa9ff24638eab3508a (127.3.139.69:45345): Couldn't send request to peer 51e82953ed2247fa9ff24638eab3508a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a12a084dba23425996e9122c083c1444. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:02.623484 11957 raft_consensus.cc:2932] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } } }
I20240502 07:59:02.623833 9908 raft_consensus.cc:2932] T a12a084dba23425996e9122c083c1444 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } } }
Runtime error: ksck discovered errors
I20240502 07:59:02.627526 9776 raft_consensus.cc:2932] T a12a084dba23425996e9122c083c1444 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } } }
I20240502 07:59:02.631336 9370 catalog_manager.cc:5484] T a12a084dba23425996e9122c083c1444 P 6cc5b9962ed24441bcba0c07b9f5438e reported cstate change: config changed from index -1 to 4, NON_VOTER 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) added. New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } } }
I20240502 07:59:02.640144 9646 consensus_queue.cc:237] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } }
I20240502 07:59:02.641783 9776 raft_consensus.cc:1261] T ecd0995c31fe4e708712c62ca60511c9 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:02.642190 11573 consensus_queue.cc:1035] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:02.642956 9908 raft_consensus.cc:1261] T ecd0995c31fe4e708712c62ca60511c9 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:02.643373 11913 consensus_queue.cc:1035] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:02.645021 11913 raft_consensus.cc:2932] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } } }
I20240502 07:59:02.647125 9908 raft_consensus.cc:2932] T ecd0995c31fe4e708712c62ca60511c9 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } } }
I20240502 07:59:02.647737 9369 catalog_manager.cc:5484] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a reported cstate change: config changed from index -1 to 4, NON_VOTER 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) added. New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:02.648597 9776 raft_consensus.cc:2932] T ecd0995c31fe4e708712c62ca60511c9 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } } }
W20240502 07:59:02.650224 9581 consensus_peers.cc:485] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a -> Peer 98bef2e94ade4763a66c1ad68259744a (127.3.139.71:46577): Couldn't send request to peer 98bef2e94ade4763a66c1ad68259744a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ecd0995c31fe4e708712c62ca60511c9. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:02.659500 9908 consensus_queue.cc:237] T 0fab37c27e184a2cacb08f05457fa6ae P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } }
I20240502 07:59:02.661155 9515 raft_consensus.cc:1261] T 0fab37c27e184a2cacb08f05457fa6ae P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:02.661521 9646 raft_consensus.cc:1261] T 0fab37c27e184a2cacb08f05457fa6ae P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:02.661661 11979 consensus_queue.cc:1035] T 0fab37c27e184a2cacb08f05457fa6ae P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:02.662739 11972 consensus_queue.cc:1035] T 0fab37c27e184a2cacb08f05457fa6ae P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:02.664407 11972 raft_consensus.cc:2932] T 0fab37c27e184a2cacb08f05457fa6ae P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } } }
W20240502 07:59:02.666093 9843 consensus_peers.cc:485] T 0fab37c27e184a2cacb08f05457fa6ae P 6cc5b9962ed24441bcba0c07b9f5438e -> Peer a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72:36271): Couldn't send request to peer a95f1d3af33f4c879fd290ca917d6329. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0fab37c27e184a2cacb08f05457fa6ae. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:02.666805 9367 catalog_manager.cc:5484] T 0fab37c27e184a2cacb08f05457fa6ae P 6cc5b9962ed24441bcba0c07b9f5438e reported cstate change: config changed from index -1 to 4, NON_VOTER a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) added. New cstate: current_term: 1 leader_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:02.670100 9646 raft_consensus.cc:2932] T 0fab37c27e184a2cacb08f05457fa6ae P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } } }
I20240502 07:59:02.670981 9515 raft_consensus.cc:2932] T 0fab37c27e184a2cacb08f05457fa6ae P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } } }
I20240502 07:59:02.679760 9908 consensus_queue.cc:237] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } }
I20240502 07:59:02.681138 9776 raft_consensus.cc:1261] T af07c527fff94115a743df1c73c3f4b5 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:02.681135 9515 raft_consensus.cc:1261] T af07c527fff94115a743df1c73c3f4b5 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:02.681545 11972 consensus_queue.cc:1035] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:02.681838 11972 consensus_queue.cc:1035] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 07:59:02.685513 9842 consensus_peers.cc:485] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e -> Peer 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70:45329): Couldn't send request to peer 8eb9945f989d4be6b51e1531a1365350. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: af07c527fff94115a743df1c73c3f4b5. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:02.685894 11972 raft_consensus.cc:2932] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } } }
I20240502 07:59:02.687467 9776 raft_consensus.cc:2932] T af07c527fff94115a743df1c73c3f4b5 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } } }
I20240502 07:59:02.688032 9367 catalog_manager.cc:5484] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e reported cstate change: config changed from index -1 to 4, NON_VOTER 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) added. New cstate: current_term: 1 leader_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:02.692302 9515 raft_consensus.cc:2932] T af07c527fff94115a743df1c73c3f4b5 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } } }
I20240502 07:59:02.700412 9646 consensus_queue.cc:237] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } }
I20240502 07:59:02.701763 9776 raft_consensus.cc:1261] T 364d227472894712a2ae15f4589a5a5b P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:02.701783 9515 raft_consensus.cc:1261] T 364d227472894712a2ae15f4589a5a5b P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 07:59:02.702051 9581 consensus_peers.cc:485] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a -> Peer 98bef2e94ade4763a66c1ad68259744a (127.3.139.71:46577): Couldn't send request to peer 98bef2e94ade4763a66c1ad68259744a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 364d227472894712a2ae15f4589a5a5b. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:02.702203 11913 consensus_queue.cc:1035] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:02.702504 11913 consensus_queue.cc:1035] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:02.703958 11913 raft_consensus.cc:2932] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } } }
I20240502 07:59:02.704346 9515 raft_consensus.cc:2932] T 364d227472894712a2ae15f4589a5a5b P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } } }
I20240502 07:59:02.706560 9367 catalog_manager.cc:5484] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a reported cstate change: config changed from index -1 to 4, NON_VOTER 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) added. New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:02.709880 9776 raft_consensus.cc:2932] T 364d227472894712a2ae15f4589a5a5b P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } } }
I20240502 07:59:02.716596 9646 consensus_queue.cc:237] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } }
I20240502 07:59:02.718214 9776 raft_consensus.cc:1261] T 8ceb129e9cbe4c469e7dfba419719556 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:02.719157 9908 raft_consensus.cc:1261] T 8ceb129e9cbe4c469e7dfba419719556 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:02.719534 11573 consensus_queue.cc:1035] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:02.720702 11573 consensus_queue.cc:1035] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:02.721786 11573 raft_consensus.cc:2932] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } } }
W20240502 07:59:02.722883 9582 consensus_peers.cc:485] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a -> Peer 51e82953ed2247fa9ff24638eab3508a (127.3.139.69:45345): Couldn't send request to peer 51e82953ed2247fa9ff24638eab3508a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8ceb129e9cbe4c469e7dfba419719556. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:02.724648 9908 raft_consensus.cc:2932] T 8ceb129e9cbe4c469e7dfba419719556 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } } }
I20240502 07:59:02.724853 9776 raft_consensus.cc:2932] T 8ceb129e9cbe4c469e7dfba419719556 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } } }
I20240502 07:59:02.725781 9370 catalog_manager.cc:5484] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a reported cstate change: config changed from index -1 to 4, NON_VOTER 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) added. New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:02.741930 9776 consensus_queue.cc:237] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } }
I20240502 07:59:02.743376 9515 raft_consensus.cc:1261] T 686b5cf6614d45c18cc776bd79567667 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:02.743882 9646 raft_consensus.cc:1261] T 686b5cf6614d45c18cc776bd79567667 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:02.744230 11914 consensus_queue.cc:1035] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:02.745707 11914 consensus_queue.cc:1035] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 07:59:02.749399 9712 consensus_peers.cc:485] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 -> Peer a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72:36271): Couldn't send request to peer a95f1d3af33f4c879fd290ca917d6329. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 686b5cf6614d45c18cc776bd79567667. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:02.752056 11914 raft_consensus.cc:2932] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } } }
I20240502 07:59:02.754566 9515 raft_consensus.cc:2932] T 686b5cf6614d45c18cc776bd79567667 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } } }
I20240502 07:59:02.754860 9367 catalog_manager.cc:5484] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 reported cstate change: config changed from index -1 to 4, NON_VOTER a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) added. New cstate: current_term: 1 leader_uuid: "1472c64688914149a318c7b8850a3df3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:02.760118 9646 raft_consensus.cc:2932] T 686b5cf6614d45c18cc776bd79567667 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } } }
I20240502 07:59:02.766206 9908 consensus_queue.cc:237] T fd251de859e049c38a632a17b2fbdeae P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } }
I20240502 07:59:02.767803 9515 raft_consensus.cc:1261] T fd251de859e049c38a632a17b2fbdeae P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:02.767868 9646 raft_consensus.cc:1261] T fd251de859e049c38a632a17b2fbdeae P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:02.768158 11972 consensus_queue.cc:1035] T fd251de859e049c38a632a17b2fbdeae P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 07:59:02.768338 9842 consensus_peers.cc:485] T fd251de859e049c38a632a17b2fbdeae P 6cc5b9962ed24441bcba0c07b9f5438e -> Peer 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70:45329): Couldn't send request to peer 8eb9945f989d4be6b51e1531a1365350. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fd251de859e049c38a632a17b2fbdeae. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:02.768487 11972 consensus_queue.cc:1035] T fd251de859e049c38a632a17b2fbdeae P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:02.770190 11979 raft_consensus.cc:2932] T fd251de859e049c38a632a17b2fbdeae P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } } }
I20240502 07:59:02.770539 9515 raft_consensus.cc:2932] T fd251de859e049c38a632a17b2fbdeae P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } } }
I20240502 07:59:02.772917 9646 raft_consensus.cc:2932] T fd251de859e049c38a632a17b2fbdeae P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } } }
I20240502 07:59:02.773314 9367 catalog_manager.cc:5484] T fd251de859e049c38a632a17b2fbdeae P a201db721bda4dc3a4422dc8fa9bf72e reported cstate change: config changed from index -1 to 4, NON_VOTER 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) added. New cstate: current_term: 1 leader_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } } }
I20240502 07:59:02.784126 9646 consensus_queue.cc:237] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } }
I20240502 07:59:02.786026 9776 raft_consensus.cc:1261] T d562a250cee247a48e573b25ecf00429 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:02.786428 11913 consensus_queue.cc:1035] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:02.787254 9906 raft_consensus.cc:1261] T d562a250cee247a48e573b25ecf00429 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:02.787772 11573 consensus_queue.cc:1035] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:02.788686 11573 raft_consensus.cc:2932] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } } }
I20240502 07:59:02.791133 9370 catalog_manager.cc:5484] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a reported cstate change: config changed from index -1 to 4, NON_VOTER a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) added. New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:02.792321 9776 raft_consensus.cc:2932] T d562a250cee247a48e573b25ecf00429 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } } }
W20240502 07:59:02.796077 9581 consensus_peers.cc:485] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a -> Peer a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72:36271): Couldn't send request to peer a95f1d3af33f4c879fd290ca917d6329. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d562a250cee247a48e573b25ecf00429. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:02.796451 9906 raft_consensus.cc:2932] T d562a250cee247a48e573b25ecf00429 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } } }
I20240502 07:59:02.807337 11701 heartbeater.cc:503] Master 127.3.139.126:46479 was elected leader, sending a full tablet report...
I20240502 07:59:02.813596 9515 consensus_queue.cc:237] T 16c2ef3c0f5c44b39327175f6519fb95 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } }
I20240502 07:59:02.815121 9906 raft_consensus.cc:1261] T 16c2ef3c0f5c44b39327175f6519fb95 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:02.815285 9776 raft_consensus.cc:1261] T 16c2ef3c0f5c44b39327175f6519fb95 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:02.815729 11109 consensus_queue.cc:1035] T 16c2ef3c0f5c44b39327175f6519fb95 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:02.815958 11961 consensus_queue.cc:1035] T 16c2ef3c0f5c44b39327175f6519fb95 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:02.818253 11961 raft_consensus.cc:2932] T 16c2ef3c0f5c44b39327175f6519fb95 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } } }
I20240502 07:59:02.818572 9906 raft_consensus.cc:2932] T 16c2ef3c0f5c44b39327175f6519fb95 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } } }
I20240502 07:59:02.820416 9776 raft_consensus.cc:2932] T 16c2ef3c0f5c44b39327175f6519fb95 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } } }
I20240502 07:59:02.821290 9370 catalog_manager.cc:5484] T 16c2ef3c0f5c44b39327175f6519fb95 P 6cc5b9962ed24441bcba0c07b9f5438e reported cstate change: config changed from index -1 to 4, NON_VOTER 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) added. New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } } }
W20240502 07:59:02.822487 9450 consensus_peers.cc:485] T 16c2ef3c0f5c44b39327175f6519fb95 P a201db721bda4dc3a4422dc8fa9bf72e -> Peer 98bef2e94ade4763a66c1ad68259744a (127.3.139.71:46577): Couldn't send request to peer 98bef2e94ade4763a66c1ad68259744a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16c2ef3c0f5c44b39327175f6519fb95. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:02.830859 9776 consensus_queue.cc:237] T d3db04c54a184eb8a1c23fec9e7145eb P 1472c64688914149a318c7b8850a3df3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } }
I20240502 07:59:02.832448 9906 raft_consensus.cc:1261] T d3db04c54a184eb8a1c23fec9e7145eb P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:02.832552 9646 raft_consensus.cc:1261] T d3db04c54a184eb8a1c23fec9e7145eb P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:02.832866 11062 consensus_queue.cc:1035] T d3db04c54a184eb8a1c23fec9e7145eb P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:02.833067 11943 consensus_queue.cc:1035] T d3db04c54a184eb8a1c23fec9e7145eb P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 07:59:02.835578 9713 consensus_peers.cc:485] T d3db04c54a184eb8a1c23fec9e7145eb P 1472c64688914149a318c7b8850a3df3 -> Peer 51e82953ed2247fa9ff24638eab3508a (127.3.139.69:45345): Couldn't send request to peer 51e82953ed2247fa9ff24638eab3508a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d3db04c54a184eb8a1c23fec9e7145eb. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:02.835619 11943 raft_consensus.cc:2932] T d3db04c54a184eb8a1c23fec9e7145eb P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } } }
I20240502 07:59:02.836031 9646 raft_consensus.cc:2932] T d3db04c54a184eb8a1c23fec9e7145eb P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } } }
I20240502 07:59:02.836017 9906 raft_consensus.cc:2932] T d3db04c54a184eb8a1c23fec9e7145eb P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } } }
I20240502 07:59:02.838366 9369 catalog_manager.cc:5484] T d3db04c54a184eb8a1c23fec9e7145eb P 1472c64688914149a318c7b8850a3df3 reported cstate change: config changed from index -1 to 4, NON_VOTER 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) added. New cstate: current_term: 1 leader_uuid: "1472c64688914149a318c7b8850a3df3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20240502 07:59:02.838598 11971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:02.838836 11971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:02.842231 11971 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20240502 07:59:02.848902 9906 consensus_queue.cc:237] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } }
I20240502 07:59:02.850240 9646 raft_consensus.cc:1261] T 3255d89d51a94426a78364230c9b249b P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:02.850298 9515 raft_consensus.cc:1261] T 3255d89d51a94426a78364230c9b249b P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 07:59:02.850478 9843 consensus_peers.cc:485] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e -> Peer a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72:36271): Couldn't send request to peer a95f1d3af33f4c879fd290ca917d6329. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3255d89d51a94426a78364230c9b249b. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:02.850629 12024 consensus_queue.cc:1035] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:02.851151 12024 consensus_queue.cc:1035] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:02.852299 11972 raft_consensus.cc:2932] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } } }
I20240502 07:59:02.853382 9646 raft_consensus.cc:2932] T 3255d89d51a94426a78364230c9b249b P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } } }
I20240502 07:59:02.854971 9367 catalog_manager.cc:5484] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e reported cstate change: config changed from index -1 to 4, NON_VOTER a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) added. New cstate: current_term: 1 leader_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:02.856077 9515 raft_consensus.cc:2932] T 3255d89d51a94426a78364230c9b249b P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } } }
I20240502 07:59:02.863276 9646 consensus_queue.cc:237] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } }
I20240502 07:59:02.864967 9776 raft_consensus.cc:1261] T 50f059668c6b4205a15d2c076e2789b0 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:02.865373 11913 consensus_queue.cc:1035] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:02.866353 9906 raft_consensus.cc:1261] T 50f059668c6b4205a15d2c076e2789b0 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:02.866683 11913 consensus_queue.cc:1035] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:02.868925 11573 raft_consensus.cc:2932] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } } }
W20240502 07:59:02.869122 9580 consensus_peers.cc:485] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a -> Peer 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70:45329): Couldn't send request to peer 8eb9945f989d4be6b51e1531a1365350. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 50f059668c6b4205a15d2c076e2789b0. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:02.871101 9906 raft_consensus.cc:2932] T 50f059668c6b4205a15d2c076e2789b0 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } } }
I20240502 07:59:02.871243 9776 raft_consensus.cc:2932] T 50f059668c6b4205a15d2c076e2789b0 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } } }
I20240502 07:59:02.872951 9370 catalog_manager.cc:5484] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a reported cstate change: config changed from index -1 to 4, NON_VOTER 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) added. New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:02.880983 9646 consensus_queue.cc:237] T fcdf47a04bdd4b9fa600a0bba59a4542 P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } }
I20240502 07:59:02.882356 9906 raft_consensus.cc:1261] T fcdf47a04bdd4b9fa600a0bba59a4542 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:02.882505 9515 raft_consensus.cc:1261] T fcdf47a04bdd4b9fa600a0bba59a4542 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 07:59:02.882649 9582 consensus_peers.cc:485] T fcdf47a04bdd4b9fa600a0bba59a4542 P 8337af1943564c0dad18b45a597b805a -> Peer 51e82953ed2247fa9ff24638eab3508a (127.3.139.69:45345): Couldn't send request to peer 51e82953ed2247fa9ff24638eab3508a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fcdf47a04bdd4b9fa600a0bba59a4542. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:02.882943 11997 consensus_queue.cc:1035] T fcdf47a04bdd4b9fa600a0bba59a4542 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:02.883302 11913 consensus_queue.cc:1035] T fcdf47a04bdd4b9fa600a0bba59a4542 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:02.884933 11573 raft_consensus.cc:2932] T fcdf47a04bdd4b9fa600a0bba59a4542 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } } }
I20240502 07:59:02.887178 9515 raft_consensus.cc:2932] T fcdf47a04bdd4b9fa600a0bba59a4542 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } } }
I20240502 07:59:02.888258 9367 catalog_manager.cc:5484] T fcdf47a04bdd4b9fa600a0bba59a4542 P 8337af1943564c0dad18b45a597b805a reported cstate change: config changed from index -1 to 4, NON_VOTER 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) added. New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:02.889978 9906 raft_consensus.cc:2932] T fcdf47a04bdd4b9fa600a0bba59a4542 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } } }
I20240502 07:59:02.895879 9515 consensus_queue.cc:237] T e2ba8220f81a4d6cb3611ca53db91fe1 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } }
W20240502 07:59:02.897461 9450 consensus_peers.cc:485] T e2ba8220f81a4d6cb3611ca53db91fe1 P a201db721bda4dc3a4422dc8fa9bf72e -> Peer 98bef2e94ade4763a66c1ad68259744a (127.3.139.71:46577): Couldn't send request to peer 98bef2e94ade4763a66c1ad68259744a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e2ba8220f81a4d6cb3611ca53db91fe1. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:02.897753 9646 raft_consensus.cc:1261] T e2ba8220f81a4d6cb3611ca53db91fe1 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:02.898038 9776 raft_consensus.cc:1261] T e2ba8220f81a4d6cb3611ca53db91fe1 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:02.898151 11572 consensus_queue.cc:1035] T e2ba8220f81a4d6cb3611ca53db91fe1 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:02.898448 11572 consensus_queue.cc:1035] T e2ba8220f81a4d6cb3611ca53db91fe1 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:02.900180 11109 raft_consensus.cc:2932] T e2ba8220f81a4d6cb3611ca53db91fe1 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } } }
I20240502 07:59:02.900653 9646 raft_consensus.cc:2932] T e2ba8220f81a4d6cb3611ca53db91fe1 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } } }
I20240502 07:59:02.902596 9776 raft_consensus.cc:2932] T e2ba8220f81a4d6cb3611ca53db91fe1 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } } }
I20240502 07:59:02.904803 9367 catalog_manager.cc:5484] T e2ba8220f81a4d6cb3611ca53db91fe1 P 8337af1943564c0dad18b45a597b805a reported cstate change: config changed from index -1 to 4, NON_VOTER 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) added. New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } } }
I20240502 07:59:02.912252 9906 consensus_queue.cc:237] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } }
I20240502 07:59:02.914314 9515 raft_consensus.cc:1261] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:02.914598 9776 raft_consensus.cc:1261] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:02.916944 11972 consensus_queue.cc:1035] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:02.917281 11972 consensus_queue.cc:1035] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 07:59:02.919656 9844 consensus_peers.cc:485] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e -> Peer 51e82953ed2247fa9ff24638eab3508a (127.3.139.69:45345): Couldn't send request to peer 51e82953ed2247fa9ff24638eab3508a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7da9d3b7fe8c40f6a68bb1b7a926d994. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:02.920828 12024 raft_consensus.cc:2932] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } } }
I20240502 07:59:02.923417 9776 raft_consensus.cc:2932] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } } }
I20240502 07:59:02.923717 9370 catalog_manager.cc:5484] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e reported cstate change: config changed from index -1 to 4, NON_VOTER 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) added. New cstate: current_term: 1 leader_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:02.930562 9515 raft_consensus.cc:2932] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } } }
I20240502 07:59:02.938297 9515 consensus_queue.cc:237] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } }
I20240502 07:59:02.939702 9776 raft_consensus.cc:1261] T bc896a65284c48cd864bb6b57dbe5ca0 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:02.939705 9646 raft_consensus.cc:1261] T bc896a65284c48cd864bb6b57dbe5ca0 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:02.940039 11957 consensus_queue.cc:1035] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:02.941267 11957 consensus_queue.cc:1035] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:02.941926 11957 raft_consensus.cc:2932] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } } }
I20240502 07:59:02.943465 9646 raft_consensus.cc:2932] T bc896a65284c48cd864bb6b57dbe5ca0 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } } }
I20240502 07:59:02.945716 9369 catalog_manager.cc:5484] T bc896a65284c48cd864bb6b57dbe5ca0 P 8337af1943564c0dad18b45a597b805a reported cstate change: config changed from index -1 to 4, NON_VOTER a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) added. New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } } }
W20240502 07:59:02.946482 9450 consensus_peers.cc:485] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e -> Peer a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72:36271): Couldn't send request to peer a95f1d3af33f4c879fd290ca917d6329. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bc896a65284c48cd864bb6b57dbe5ca0. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:02.958830 9776 raft_consensus.cc:2932] T bc896a65284c48cd864bb6b57dbe5ca0 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } } }
I20240502 07:59:02.963364 9515 consensus_queue.cc:237] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } }
I20240502 07:59:02.964895 9646 raft_consensus.cc:1261] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:02.965384 9906 raft_consensus.cc:1261] T eddeb6a23ec1418eb35b414f5d52e7b7 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:02.965915 11961 consensus_queue.cc:1035] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:02.966203 11961 consensus_queue.cc:1035] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 07:59:02.969123 9449 consensus_peers.cc:485] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e -> Peer 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70:45329): Couldn't send request to peer 8eb9945f989d4be6b51e1531a1365350. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: eddeb6a23ec1418eb35b414f5d52e7b7. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:02.969429 11109 raft_consensus.cc:2932] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } } }
I20240502 07:59:02.969791 9646 raft_consensus.cc:2932] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } } }
I20240502 07:59:02.971997 9366 catalog_manager.cc:5484] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e reported cstate change: config changed from index -1 to 4, NON_VOTER 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) added. New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:02.976253 11240 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:02.977274 9624 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:02.979737 11371 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:02.982590 9757 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:02.983518 11502 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:02.991540 9886 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:03.009439 9906 raft_consensus.cc:2932] T eddeb6a23ec1418eb35b414f5d52e7b7 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } } }
I20240502 07:59:03.010480 9493 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:03.021914 11635 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:03.025609 9515 consensus_queue.cc:237] T 140868fcb19a4e7c837ce0c2c1c7041e P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } }
I20240502 07:59:03.027225 9776 raft_consensus.cc:1261] T 140868fcb19a4e7c837ce0c2c1c7041e P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:03.027772 9646 raft_consensus.cc:1261] T 140868fcb19a4e7c837ce0c2c1c7041e P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:03.028087 11572 consensus_queue.cc:1035] T 140868fcb19a4e7c837ce0c2c1c7041e P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:03.028239 11961 consensus_queue.cc:1035] T 140868fcb19a4e7c837ce0c2c1c7041e P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 07:59:03.028781 9450 consensus_peers.cc:485] T 140868fcb19a4e7c837ce0c2c1c7041e P a201db721bda4dc3a4422dc8fa9bf72e -> Peer 98bef2e94ade4763a66c1ad68259744a (127.3.139.71:46577): Couldn't send request to peer 98bef2e94ade4763a66c1ad68259744a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 140868fcb19a4e7c837ce0c2c1c7041e. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:03.031075 11957 raft_consensus.cc:2932] T 140868fcb19a4e7c837ce0c2c1c7041e P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } } }
I20240502 07:59:03.031507 9776 raft_consensus.cc:2932] T 140868fcb19a4e7c837ce0c2c1c7041e P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } } }
I20240502 07:59:03.033360 9646 raft_consensus.cc:2932] T 140868fcb19a4e7c837ce0c2c1c7041e P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } } }
I20240502 07:59:03.033850 9366 catalog_manager.cc:5484] T 140868fcb19a4e7c837ce0c2c1c7041e P a201db721bda4dc3a4422dc8fa9bf72e reported cstate change: config changed from index -1 to 4, NON_VOTER 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) added. New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
Runtime error: ksck discovered errors
I20240502 07:59:03.075397 12198 ts_tablet_manager.cc:902] T 92f37161960e4ee894c75d9034434001 P 8eb9945f989d4be6b51e1531a1365350: Initiating tablet copy from peer 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191)
I20240502 07:59:03.077637 12198 tablet_copy_client.cc:308] T 92f37161960e4ee894c75d9034434001 P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.67:43191
I20240502 07:59:03.091324 9797 tablet_copy_service.cc:140] P 1472c64688914149a318c7b8850a3df3: Received BeginTabletCopySession request for tablet 92f37161960e4ee894c75d9034434001 from peer 8eb9945f989d4be6b51e1531a1365350 ({username='slave'} at 127.3.139.70:41973)
I20240502 07:59:03.091470 9797 tablet_copy_service.cc:161] P 1472c64688914149a318c7b8850a3df3: Beginning new tablet copy session on tablet 92f37161960e4ee894c75d9034434001 from peer 8eb9945f989d4be6b51e1531a1365350 at {username='slave'} at 127.3.139.70:41973: session id = 8eb9945f989d4be6b51e1531a1365350-92f37161960e4ee894c75d9034434001
I20240502 07:59:03.092842 9797 tablet_copy_source_session.cc:205] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:03.096122 12198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92f37161960e4ee894c75d9034434001. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:03.105147 12198 tablet_copy_client.cc:783] T 92f37161960e4ee894c75d9034434001 P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:03.105415 12198 tablet_copy_client.cc:647] T 92f37161960e4ee894c75d9034434001 P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:03.107025 12200 tablet_copy_client.cc:1002] T 92f37161960e4ee894c75d9034434001 P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Injecting latency into file download: 1500
I20240502 07:59:03.121549 12231 ts_tablet_manager.cc:902] T a12a084dba23425996e9122c083c1444 P 51e82953ed2247fa9ff24638eab3508a: Initiating tablet copy from peer a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551)
I20240502 07:59:03.122174 12231 tablet_copy_client.cc:308] T a12a084dba23425996e9122c083c1444 P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.65:39551
I20240502 07:59:03.126935 9535 tablet_copy_service.cc:140] P a201db721bda4dc3a4422dc8fa9bf72e: Received BeginTabletCopySession request for tablet a12a084dba23425996e9122c083c1444 from peer 51e82953ed2247fa9ff24638eab3508a ({username='slave'} at 127.3.139.69:55681)
I20240502 07:59:03.127086 9535 tablet_copy_service.cc:161] P a201db721bda4dc3a4422dc8fa9bf72e: Beginning new tablet copy session on tablet a12a084dba23425996e9122c083c1444 from peer 51e82953ed2247fa9ff24638eab3508a at {username='slave'} at 127.3.139.69:55681: session id = 51e82953ed2247fa9ff24638eab3508a-a12a084dba23425996e9122c083c1444
I20240502 07:59:03.128311 9535 tablet_copy_source_session.cc:205] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:03.132133 12231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a12a084dba23425996e9122c083c1444. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:03.137346 12231 tablet_copy_client.cc:783] T a12a084dba23425996e9122c083c1444 P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:03.137612 12231 tablet_copy_client.cc:647] T a12a084dba23425996e9122c083c1444 P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:03.139179 12232 tablet_copy_client.cc:1002] T a12a084dba23425996e9122c083c1444 P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Injecting latency into file download: 1500
I20240502 07:59:03.161736 12267 ts_tablet_manager.cc:902] T 8ceb129e9cbe4c469e7dfba419719556 P 51e82953ed2247fa9ff24638eab3508a: Initiating tablet copy from peer 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667)
I20240502 07:59:03.162262 12267 tablet_copy_client.cc:308] T 8ceb129e9cbe4c469e7dfba419719556 P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.66:46667
I20240502 07:59:03.165397 9666 tablet_copy_service.cc:140] P 8337af1943564c0dad18b45a597b805a: Received BeginTabletCopySession request for tablet 8ceb129e9cbe4c469e7dfba419719556 from peer 51e82953ed2247fa9ff24638eab3508a ({username='slave'} at 127.3.139.69:35563)
I20240502 07:59:03.165509 9666 tablet_copy_service.cc:161] P 8337af1943564c0dad18b45a597b805a: Beginning new tablet copy session on tablet 8ceb129e9cbe4c469e7dfba419719556 from peer 51e82953ed2247fa9ff24638eab3508a at {username='slave'} at 127.3.139.69:35563: session id = 51e82953ed2247fa9ff24638eab3508a-8ceb129e9cbe4c469e7dfba419719556
I20240502 07:59:03.166507 9666 tablet_copy_source_session.cc:205] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:03.167551 12267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ceb129e9cbe4c469e7dfba419719556. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:03.169203 12267 tablet_copy_client.cc:783] T 8ceb129e9cbe4c469e7dfba419719556 P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:03.169356 12267 tablet_copy_client.cc:647] T 8ceb129e9cbe4c469e7dfba419719556 P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:03.170454 12270 tablet_copy_client.cc:1002] T 8ceb129e9cbe4c469e7dfba419719556 P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Injecting latency into file download: 1500
I20240502 07:59:03.182652 12277 ts_tablet_manager.cc:902] T af07c527fff94115a743df1c73c3f4b5 P 8eb9945f989d4be6b51e1531a1365350: Initiating tablet copy from peer 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379)
I20240502 07:59:03.183142 12277 tablet_copy_client.cc:308] T af07c527fff94115a743df1c73c3f4b5 P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.68:37379
I20240502 07:59:03.186652 9928 tablet_copy_service.cc:140] P 6cc5b9962ed24441bcba0c07b9f5438e: Received BeginTabletCopySession request for tablet af07c527fff94115a743df1c73c3f4b5 from peer 8eb9945f989d4be6b51e1531a1365350 ({username='slave'} at 127.3.139.70:41665)
I20240502 07:59:03.186770 9928 tablet_copy_service.cc:161] P 6cc5b9962ed24441bcba0c07b9f5438e: Beginning new tablet copy session on tablet af07c527fff94115a743df1c73c3f4b5 from peer 8eb9945f989d4be6b51e1531a1365350 at {username='slave'} at 127.3.139.70:41665: session id = 8eb9945f989d4be6b51e1531a1365350-af07c527fff94115a743df1c73c3f4b5
I20240502 07:59:03.187835 9928 tablet_copy_source_session.cc:205] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:03.188933 12277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af07c527fff94115a743df1c73c3f4b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:03.190385 12285 ts_tablet_manager.cc:902] T fd251de859e049c38a632a17b2fbdeae P 8eb9945f989d4be6b51e1531a1365350: Initiating tablet copy from peer 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379)
I20240502 07:59:03.190876 12285 tablet_copy_client.cc:308] T fd251de859e049c38a632a17b2fbdeae P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.68:37379
I20240502 07:59:03.191012 12277 tablet_copy_client.cc:783] T af07c527fff94115a743df1c73c3f4b5 P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:03.191181 12277 tablet_copy_client.cc:647] T af07c527fff94115a743df1c73c3f4b5 P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:03.191418 9928 tablet_copy_service.cc:140] P 6cc5b9962ed24441bcba0c07b9f5438e: Received BeginTabletCopySession request for tablet fd251de859e049c38a632a17b2fbdeae from peer 8eb9945f989d4be6b51e1531a1365350 ({username='slave'} at 127.3.139.70:41665)
I20240502 07:59:03.191495 9928 tablet_copy_service.cc:161] P 6cc5b9962ed24441bcba0c07b9f5438e: Beginning new tablet copy session on tablet fd251de859e049c38a632a17b2fbdeae from peer 8eb9945f989d4be6b51e1531a1365350 at {username='slave'} at 127.3.139.70:41665: session id = 8eb9945f989d4be6b51e1531a1365350-fd251de859e049c38a632a17b2fbdeae
I20240502 07:59:03.192131 9928 tablet_copy_source_session.cc:205] T fd251de859e049c38a632a17b2fbdeae P 6cc5b9962ed24441bcba0c07b9f5438e: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:03.192580 12281 tablet_copy_client.cc:1002] T af07c527fff94115a743df1c73c3f4b5 P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Injecting latency into file download: 1500
W20240502 07:59:03.192580 12192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
I20240502 07:59:03.192797 12285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd251de859e049c38a632a17b2fbdeae. 1 dirs total, 0 dirs full, 0 dirs failed
W20240502 07:59:03.192804 12192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20240502 07:59:03.194394 12285 tablet_copy_client.cc:783] T fd251de859e049c38a632a17b2fbdeae P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:03.194506 12285 tablet_copy_client.cc:647] T fd251de859e049c38a632a17b2fbdeae P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:03.195596 12287 tablet_copy_client.cc:1002] T fd251de859e049c38a632a17b2fbdeae P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Injecting latency into file download: 1500
W20240502 07:59:03.196208 12192 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20240502 07:59:03.213867 12297 ts_tablet_manager.cc:902] T ecd0995c31fe4e708712c62ca60511c9 P 98bef2e94ade4763a66c1ad68259744a: Initiating tablet copy from peer 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667)
I20240502 07:59:03.214565 12297 tablet_copy_client.cc:308] T ecd0995c31fe4e708712c62ca60511c9 P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.66:46667
I20240502 07:59:03.217876 9666 tablet_copy_service.cc:140] P 8337af1943564c0dad18b45a597b805a: Received BeginTabletCopySession request for tablet ecd0995c31fe4e708712c62ca60511c9 from peer 98bef2e94ade4763a66c1ad68259744a ({username='slave'} at 127.3.139.71:52353)
I20240502 07:59:03.217978 9666 tablet_copy_service.cc:161] P 8337af1943564c0dad18b45a597b805a: Beginning new tablet copy session on tablet ecd0995c31fe4e708712c62ca60511c9 from peer 98bef2e94ade4763a66c1ad68259744a at {username='slave'} at 127.3.139.71:52353: session id = 98bef2e94ade4763a66c1ad68259744a-ecd0995c31fe4e708712c62ca60511c9
I20240502 07:59:03.218866 9666 tablet_copy_source_session.cc:205] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:03.221582 12297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecd0995c31fe4e708712c62ca60511c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:03.226336 12303 ts_tablet_manager.cc:902] T 0fab37c27e184a2cacb08f05457fa6ae P a95f1d3af33f4c879fd290ca917d6329: Initiating tablet copy from peer 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379)
I20240502 07:59:03.226761 12297 tablet_copy_client.cc:783] T ecd0995c31fe4e708712c62ca60511c9 P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:03.226933 12297 tablet_copy_client.cc:647] T ecd0995c31fe4e708712c62ca60511c9 P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:03.227005 12303 tablet_copy_client.cc:308] T 0fab37c27e184a2cacb08f05457fa6ae P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.68:37379
I20240502 07:59:03.228330 12301 tablet_copy_client.cc:1002] T ecd0995c31fe4e708712c62ca60511c9 P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Injecting latency into file download: 1500
I20240502 07:59:03.231074 9928 tablet_copy_service.cc:140] P 6cc5b9962ed24441bcba0c07b9f5438e: Received BeginTabletCopySession request for tablet 0fab37c27e184a2cacb08f05457fa6ae from peer a95f1d3af33f4c879fd290ca917d6329 ({username='slave'} at 127.3.139.72:51283)
I20240502 07:59:03.231178 9928 tablet_copy_service.cc:161] P 6cc5b9962ed24441bcba0c07b9f5438e: Beginning new tablet copy session on tablet 0fab37c27e184a2cacb08f05457fa6ae from peer a95f1d3af33f4c879fd290ca917d6329 at {username='slave'} at 127.3.139.72:51283: session id = a95f1d3af33f4c879fd290ca917d6329-0fab37c27e184a2cacb08f05457fa6ae
I20240502 07:59:03.231969 9928 tablet_copy_source_session.cc:205] T 0fab37c27e184a2cacb08f05457fa6ae P 6cc5b9962ed24441bcba0c07b9f5438e: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:03.234961 12303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fab37c27e184a2cacb08f05457fa6ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:03.239986 12303 tablet_copy_client.cc:783] T 0fab37c27e184a2cacb08f05457fa6ae P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:03.240144 12303 tablet_copy_client.cc:647] T 0fab37c27e184a2cacb08f05457fa6ae P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:03.241338 12304 tablet_copy_client.cc:1002] T 0fab37c27e184a2cacb08f05457fa6ae P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Injecting latency into file download: 1500
I20240502 07:59:03.297581 9757 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:03.299306 9886 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:03.302495 11502 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:03.303905 11371 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:03.313903 11240 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:03.316161 9624 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:03.324970 12341 ts_tablet_manager.cc:902] T 686b5cf6614d45c18cc776bd79567667 P a95f1d3af33f4c879fd290ca917d6329: Initiating tablet copy from peer 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191)
I20240502 07:59:03.325608 12341 tablet_copy_client.cc:308] T 686b5cf6614d45c18cc776bd79567667 P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.67:43191
I20240502 07:59:03.333602 12343 ts_tablet_manager.cc:902] T 364d227472894712a2ae15f4589a5a5b P 98bef2e94ade4763a66c1ad68259744a: Initiating tablet copy from peer 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667)
I20240502 07:59:03.334156 9493 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:03.334153 12343 tablet_copy_client.cc:308] T 364d227472894712a2ae15f4589a5a5b P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.66:46667
I20240502 07:59:03.337352 9666 tablet_copy_service.cc:140] P 8337af1943564c0dad18b45a597b805a: Received BeginTabletCopySession request for tablet 364d227472894712a2ae15f4589a5a5b from peer 98bef2e94ade4763a66c1ad68259744a ({username='slave'} at 127.3.139.71:52353)
I20240502 07:59:03.337448 9666 tablet_copy_service.cc:161] P 8337af1943564c0dad18b45a597b805a: Beginning new tablet copy session on tablet 364d227472894712a2ae15f4589a5a5b from peer 98bef2e94ade4763a66c1ad68259744a at {username='slave'} at 127.3.139.71:52353: session id = 98bef2e94ade4763a66c1ad68259744a-364d227472894712a2ae15f4589a5a5b
I20240502 07:59:03.338326 9666 tablet_copy_source_session.cc:205] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:03.339171 12343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 364d227472894712a2ae15f4589a5a5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:03.335294 11635 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:03.344511 12343 tablet_copy_client.cc:783] T 364d227472894712a2ae15f4589a5a5b P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:03.344686 12343 tablet_copy_client.cc:647] T 364d227472894712a2ae15f4589a5a5b P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:03.345649 12351 tablet_copy_client.cc:1002] T 364d227472894712a2ae15f4589a5a5b P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Injecting latency into file download: 1500
I20240502 07:59:03.347162 12342 ts_tablet_manager.cc:902] T d562a250cee247a48e573b25ecf00429 P a95f1d3af33f4c879fd290ca917d6329: Initiating tablet copy from peer 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667)
I20240502 07:59:03.350646 12342 tablet_copy_client.cc:308] T d562a250cee247a48e573b25ecf00429 P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.66:46667
I20240502 07:59:03.351320 9797 tablet_copy_service.cc:140] P 1472c64688914149a318c7b8850a3df3: Received BeginTabletCopySession request for tablet 686b5cf6614d45c18cc776bd79567667 from peer a95f1d3af33f4c879fd290ca917d6329 ({username='slave'} at 127.3.139.72:51819)
I20240502 07:59:03.351421 9797 tablet_copy_service.cc:161] P 1472c64688914149a318c7b8850a3df3: Beginning new tablet copy session on tablet 686b5cf6614d45c18cc776bd79567667 from peer a95f1d3af33f4c879fd290ca917d6329 at {username='slave'} at 127.3.139.72:51819: session id = a95f1d3af33f4c879fd290ca917d6329-686b5cf6614d45c18cc776bd79567667
I20240502 07:59:03.352311 9797 tablet_copy_source_session.cc:205] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:03.353192 12341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 686b5cf6614d45c18cc776bd79567667. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:03.354815 9666 tablet_copy_service.cc:140] P 8337af1943564c0dad18b45a597b805a: Received BeginTabletCopySession request for tablet d562a250cee247a48e573b25ecf00429 from peer a95f1d3af33f4c879fd290ca917d6329 ({username='slave'} at 127.3.139.72:42289)
I20240502 07:59:03.354904 9666 tablet_copy_service.cc:161] P 8337af1943564c0dad18b45a597b805a: Beginning new tablet copy session on tablet d562a250cee247a48e573b25ecf00429 from peer a95f1d3af33f4c879fd290ca917d6329 at {username='slave'} at 127.3.139.72:42289: session id = a95f1d3af33f4c879fd290ca917d6329-d562a250cee247a48e573b25ecf00429
I20240502 07:59:03.355643 12341 tablet_copy_client.cc:783] T 686b5cf6614d45c18cc776bd79567667 P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:03.355767 9666 tablet_copy_source_session.cc:205] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:03.355790 12341 tablet_copy_client.cc:647] T 686b5cf6614d45c18cc776bd79567667 P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:03.356567 12342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d562a250cee247a48e573b25ecf00429. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:03.358889 12342 tablet_copy_client.cc:783] T d562a250cee247a48e573b25ecf00429 P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:03.359071 12342 tablet_copy_client.cc:647] T d562a250cee247a48e573b25ecf00429 P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:03.360527 12354 tablet_copy_client.cc:1002] T d562a250cee247a48e573b25ecf00429 P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Injecting latency into file download: 1500
I20240502 07:59:03.363618 12345 tablet_copy_client.cc:1002] T 686b5cf6614d45c18cc776bd79567667 P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Injecting latency into file download: 1500
I20240502 07:59:03.369467 12356 ts_tablet_manager.cc:902] T 50f059668c6b4205a15d2c076e2789b0 P 8eb9945f989d4be6b51e1531a1365350: Initiating tablet copy from peer 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667)
I20240502 07:59:03.370051 12356 tablet_copy_client.cc:308] T 50f059668c6b4205a15d2c076e2789b0 P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.66:46667
I20240502 07:59:03.370849 12361 ts_tablet_manager.cc:902] T d3db04c54a184eb8a1c23fec9e7145eb P 51e82953ed2247fa9ff24638eab3508a: Initiating tablet copy from peer 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191)
I20240502 07:59:03.371475 12361 tablet_copy_client.cc:308] T d3db04c54a184eb8a1c23fec9e7145eb P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.67:43191
I20240502 07:59:03.374341 9666 tablet_copy_service.cc:140] P 8337af1943564c0dad18b45a597b805a: Received BeginTabletCopySession request for tablet 50f059668c6b4205a15d2c076e2789b0 from peer 8eb9945f989d4be6b51e1531a1365350 ({username='slave'} at 127.3.139.70:48793)
I20240502 07:59:03.374429 9666 tablet_copy_service.cc:161] P 8337af1943564c0dad18b45a597b805a: Beginning new tablet copy session on tablet 50f059668c6b4205a15d2c076e2789b0 from peer 8eb9945f989d4be6b51e1531a1365350 at {username='slave'} at 127.3.139.70:48793: session id = 8eb9945f989d4be6b51e1531a1365350-50f059668c6b4205a15d2c076e2789b0
I20240502 07:59:03.375255 9666 tablet_copy_source_session.cc:205] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:03.377306 9797 tablet_copy_service.cc:140] P 1472c64688914149a318c7b8850a3df3: Received BeginTabletCopySession request for tablet d3db04c54a184eb8a1c23fec9e7145eb from peer 51e82953ed2247fa9ff24638eab3508a ({username='slave'} at 127.3.139.69:36679)
I20240502 07:59:03.377418 9797 tablet_copy_service.cc:161] P 1472c64688914149a318c7b8850a3df3: Beginning new tablet copy session on tablet d3db04c54a184eb8a1c23fec9e7145eb from peer 51e82953ed2247fa9ff24638eab3508a at {username='slave'} at 127.3.139.69:36679: session id = 51e82953ed2247fa9ff24638eab3508a-d3db04c54a184eb8a1c23fec9e7145eb
I20240502 07:59:03.377914 12366 ts_tablet_manager.cc:902] T fcdf47a04bdd4b9fa600a0bba59a4542 P 51e82953ed2247fa9ff24638eab3508a: Initiating tablet copy from peer 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667)
I20240502 07:59:03.378229 9797 tablet_copy_source_session.cc:205] T d3db04c54a184eb8a1c23fec9e7145eb P 1472c64688914149a318c7b8850a3df3: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:03.378412 12366 tablet_copy_client.cc:308] T fcdf47a04bdd4b9fa600a0bba59a4542 P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.66:46667
I20240502 07:59:03.378778 9666 tablet_copy_service.cc:140] P 8337af1943564c0dad18b45a597b805a: Received BeginTabletCopySession request for tablet fcdf47a04bdd4b9fa600a0bba59a4542 from peer 51e82953ed2247fa9ff24638eab3508a ({username='slave'} at 127.3.139.69:35563)
I20240502 07:59:03.378865 9666 tablet_copy_service.cc:161] P 8337af1943564c0dad18b45a597b805a: Beginning new tablet copy session on tablet fcdf47a04bdd4b9fa600a0bba59a4542 from peer 51e82953ed2247fa9ff24638eab3508a at {username='slave'} at 127.3.139.69:35563: session id = 51e82953ed2247fa9ff24638eab3508a-fcdf47a04bdd4b9fa600a0bba59a4542
I20240502 07:59:03.379145 12361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3db04c54a184eb8a1c23fec9e7145eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:03.379462 12356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50f059668c6b4205a15d2c076e2789b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:03.379598 9666 tablet_copy_source_session.cc:205] T fcdf47a04bdd4b9fa600a0bba59a4542 P 8337af1943564c0dad18b45a597b805a: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:03.381237 12361 tablet_copy_client.cc:783] T d3db04c54a184eb8a1c23fec9e7145eb P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:03.381366 12361 tablet_copy_client.cc:647] T d3db04c54a184eb8a1c23fec9e7145eb P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:03.382725 12362 tablet_copy_client.cc:1002] T d3db04c54a184eb8a1c23fec9e7145eb P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Injecting latency into file download: 1500
I20240502 07:59:03.382911 12366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcdf47a04bdd4b9fa600a0bba59a4542. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:03.384958 12366 tablet_copy_client.cc:783] T fcdf47a04bdd4b9fa600a0bba59a4542 P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:03.385109 12366 tablet_copy_client.cc:647] T fcdf47a04bdd4b9fa600a0bba59a4542 P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:03.385571 12356 tablet_copy_client.cc:783] T 50f059668c6b4205a15d2c076e2789b0 P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:03.385730 12356 tablet_copy_client.cc:647] T 50f059668c6b4205a15d2c076e2789b0 P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:03.386989 12360 tablet_copy_client.cc:1002] T 50f059668c6b4205a15d2c076e2789b0 P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Injecting latency into file download: 1500
Runtime error: ksck discovered errors
I20240502 07:59:03.388609 12368 tablet_copy_client.cc:1002] T fcdf47a04bdd4b9fa600a0bba59a4542 P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Injecting latency into file download: 1500
I20240502 07:59:03.395675 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.73:0
--local_ip_for_outbound_sockets=127.3.139.73
--webserver_interface=127.3.139.73
--webserver_port=0
--tserver_master_addrs=127.3.139.126:46479
--builtin_ntp_servers=127.3.139.84:39259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240502 07:59:03.412014 12397 ts_tablet_manager.cc:902] T 3255d89d51a94426a78364230c9b249b P a95f1d3af33f4c879fd290ca917d6329: Initiating tablet copy from peer 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379)
I20240502 07:59:03.412542 12397 tablet_copy_client.cc:308] T 3255d89d51a94426a78364230c9b249b P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.68:37379
I20240502 07:59:03.413067 9928 tablet_copy_service.cc:140] P 6cc5b9962ed24441bcba0c07b9f5438e: Received BeginTabletCopySession request for tablet 3255d89d51a94426a78364230c9b249b from peer a95f1d3af33f4c879fd290ca917d6329 ({username='slave'} at 127.3.139.72:51283)
I20240502 07:59:03.413169 9928 tablet_copy_service.cc:161] P 6cc5b9962ed24441bcba0c07b9f5438e: Beginning new tablet copy session on tablet 3255d89d51a94426a78364230c9b249b from peer a95f1d3af33f4c879fd290ca917d6329 at {username='slave'} at 127.3.139.72:51283: session id = a95f1d3af33f4c879fd290ca917d6329-3255d89d51a94426a78364230c9b249b
I20240502 07:59:03.413877 9928 tablet_copy_source_session.cc:205] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:03.415153 12397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3255d89d51a94426a78364230c9b249b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:03.418807 12397 tablet_copy_client.cc:783] T 3255d89d51a94426a78364230c9b249b P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:03.418953 12397 tablet_copy_client.cc:647] T 3255d89d51a94426a78364230c9b249b P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:03.420131 12405 tablet_copy_client.cc:1002] T 3255d89d51a94426a78364230c9b249b P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Injecting latency into file download: 1500
I20240502 07:59:03.428494 12417 ts_tablet_manager.cc:902] T bc896a65284c48cd864bb6b57dbe5ca0 P a95f1d3af33f4c879fd290ca917d6329: Initiating tablet copy from peer a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551)
I20240502 07:59:03.429056 12417 tablet_copy_client.cc:308] T bc896a65284c48cd864bb6b57dbe5ca0 P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.65:39551
I20240502 07:59:03.432763 9535 tablet_copy_service.cc:140] P a201db721bda4dc3a4422dc8fa9bf72e: Received BeginTabletCopySession request for tablet bc896a65284c48cd864bb6b57dbe5ca0 from peer a95f1d3af33f4c879fd290ca917d6329 ({username='slave'} at 127.3.139.72:38421)
I20240502 07:59:03.432852 9535 tablet_copy_service.cc:161] P a201db721bda4dc3a4422dc8fa9bf72e: Beginning new tablet copy session on tablet bc896a65284c48cd864bb6b57dbe5ca0 from peer a95f1d3af33f4c879fd290ca917d6329 at {username='slave'} at 127.3.139.72:38421: session id = a95f1d3af33f4c879fd290ca917d6329-bc896a65284c48cd864bb6b57dbe5ca0
I20240502 07:59:03.433531 9535 tablet_copy_source_session.cc:205] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:03.434209 12417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc896a65284c48cd864bb6b57dbe5ca0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:03.435437 12422 ts_tablet_manager.cc:902] T 16c2ef3c0f5c44b39327175f6519fb95 P 98bef2e94ade4763a66c1ad68259744a: Initiating tablet copy from peer a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551)
I20240502 07:59:03.436038 12417 tablet_copy_client.cc:783] T bc896a65284c48cd864bb6b57dbe5ca0 P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:03.436028 12422 tablet_copy_client.cc:308] T 16c2ef3c0f5c44b39327175f6519fb95 P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.65:39551
I20240502 07:59:03.436161 12417 tablet_copy_client.cc:647] T bc896a65284c48cd864bb6b57dbe5ca0 P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:03.437353 12418 tablet_copy_client.cc:1002] T bc896a65284c48cd864bb6b57dbe5ca0 P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Injecting latency into file download: 1500
I20240502 07:59:03.439323 9535 tablet_copy_service.cc:140] P a201db721bda4dc3a4422dc8fa9bf72e: Received BeginTabletCopySession request for tablet 16c2ef3c0f5c44b39327175f6519fb95 from peer 98bef2e94ade4763a66c1ad68259744a ({username='slave'} at 127.3.139.71:43779)
I20240502 07:59:03.439407 9535 tablet_copy_service.cc:161] P a201db721bda4dc3a4422dc8fa9bf72e: Beginning new tablet copy session on tablet 16c2ef3c0f5c44b39327175f6519fb95 from peer 98bef2e94ade4763a66c1ad68259744a at {username='slave'} at 127.3.139.71:43779: session id = 98bef2e94ade4763a66c1ad68259744a-16c2ef3c0f5c44b39327175f6519fb95
I20240502 07:59:03.440124 9535 tablet_copy_source_session.cc:205] T 16c2ef3c0f5c44b39327175f6519fb95 P a201db721bda4dc3a4422dc8fa9bf72e: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:03.441711 12422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16c2ef3c0f5c44b39327175f6519fb95. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:03.443935 12422 tablet_copy_client.cc:783] T 16c2ef3c0f5c44b39327175f6519fb95 P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:03.444131 12422 tablet_copy_client.cc:647] T 16c2ef3c0f5c44b39327175f6519fb95 P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:03.445200 12430 tablet_copy_client.cc:1002] T 16c2ef3c0f5c44b39327175f6519fb95 P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Injecting latency into file download: 1500
I20240502 07:59:03.456574 12442 ts_tablet_manager.cc:902] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 51e82953ed2247fa9ff24638eab3508a: Initiating tablet copy from peer 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379)
I20240502 07:59:03.457561 12442 tablet_copy_client.cc:308] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.68:37379
I20240502 07:59:03.461642 9928 tablet_copy_service.cc:140] P 6cc5b9962ed24441bcba0c07b9f5438e: Received BeginTabletCopySession request for tablet 7da9d3b7fe8c40f6a68bb1b7a926d994 from peer 51e82953ed2247fa9ff24638eab3508a ({username='slave'} at 127.3.139.69:48381)
I20240502 07:59:03.461741 9928 tablet_copy_service.cc:161] P 6cc5b9962ed24441bcba0c07b9f5438e: Beginning new tablet copy session on tablet 7da9d3b7fe8c40f6a68bb1b7a926d994 from peer 51e82953ed2247fa9ff24638eab3508a at {username='slave'} at 127.3.139.69:48381: session id = 51e82953ed2247fa9ff24638eab3508a-7da9d3b7fe8c40f6a68bb1b7a926d994
I20240502 07:59:03.462481 9928 tablet_copy_source_session.cc:205] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:03.463475 12442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7da9d3b7fe8c40f6a68bb1b7a926d994. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:03.465349 12442 tablet_copy_client.cc:783] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:03.465480 12442 tablet_copy_client.cc:647] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:03.467304 12446 tablet_copy_client.cc:1002] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Injecting latency into file download: 1500
I20240502 07:59:03.501855 12477 ts_tablet_manager.cc:902] T 140868fcb19a4e7c837ce0c2c1c7041e P 98bef2e94ade4763a66c1ad68259744a: Initiating tablet copy from peer a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551)
I20240502 07:59:03.502409 12477 tablet_copy_client.cc:308] T 140868fcb19a4e7c837ce0c2c1c7041e P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.65:39551
I20240502 07:59:03.502924 9535 tablet_copy_service.cc:140] P a201db721bda4dc3a4422dc8fa9bf72e: Received BeginTabletCopySession request for tablet 140868fcb19a4e7c837ce0c2c1c7041e from peer 98bef2e94ade4763a66c1ad68259744a ({username='slave'} at 127.3.139.71:43779)
I20240502 07:59:03.503027 9535 tablet_copy_service.cc:161] P a201db721bda4dc3a4422dc8fa9bf72e: Beginning new tablet copy session on tablet 140868fcb19a4e7c837ce0c2c1c7041e from peer 98bef2e94ade4763a66c1ad68259744a at {username='slave'} at 127.3.139.71:43779: session id = 98bef2e94ade4763a66c1ad68259744a-140868fcb19a4e7c837ce0c2c1c7041e
I20240502 07:59:03.503783 9535 tablet_copy_source_session.cc:205] T 140868fcb19a4e7c837ce0c2c1c7041e P a201db721bda4dc3a4422dc8fa9bf72e: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:03.504630 12477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 140868fcb19a4e7c837ce0c2c1c7041e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:03.507084 12477 tablet_copy_client.cc:783] T 140868fcb19a4e7c837ce0c2c1c7041e P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:03.507304 12477 tablet_copy_client.cc:647] T 140868fcb19a4e7c837ce0c2c1c7041e P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:03.508375 12479 tablet_copy_client.cc:1002] T 140868fcb19a4e7c837ce0c2c1c7041e P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Injecting latency into file download: 1500
I20240502 07:59:03.509358 12487 ts_tablet_manager.cc:902] T e2ba8220f81a4d6cb3611ca53db91fe1 P 98bef2e94ade4763a66c1ad68259744a: Initiating tablet copy from peer a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551)
I20240502 07:59:03.509805 12487 tablet_copy_client.cc:308] T e2ba8220f81a4d6cb3611ca53db91fe1 P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.65:39551
I20240502 07:59:03.510365 9535 tablet_copy_service.cc:140] P a201db721bda4dc3a4422dc8fa9bf72e: Received BeginTabletCopySession request for tablet e2ba8220f81a4d6cb3611ca53db91fe1 from peer 98bef2e94ade4763a66c1ad68259744a ({username='slave'} at 127.3.139.71:43779)
I20240502 07:59:03.510450 9535 tablet_copy_service.cc:161] P a201db721bda4dc3a4422dc8fa9bf72e: Beginning new tablet copy session on tablet e2ba8220f81a4d6cb3611ca53db91fe1 from peer 98bef2e94ade4763a66c1ad68259744a at {username='slave'} at 127.3.139.71:43779: session id = 98bef2e94ade4763a66c1ad68259744a-e2ba8220f81a4d6cb3611ca53db91fe1
I20240502 07:59:03.511317 9535 tablet_copy_source_session.cc:205] T e2ba8220f81a4d6cb3611ca53db91fe1 P a201db721bda4dc3a4422dc8fa9bf72e: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:03.512292 12487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2ba8220f81a4d6cb3611ca53db91fe1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:03.512718 12497 ts_tablet_manager.cc:902] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8eb9945f989d4be6b51e1531a1365350: Initiating tablet copy from peer a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551)
I20240502 07:59:03.513415 12497 tablet_copy_client.cc:308] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.65:39551
I20240502 07:59:03.516351 12487 tablet_copy_client.cc:783] T e2ba8220f81a4d6cb3611ca53db91fe1 P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:03.516547 12487 tablet_copy_client.cc:647] T e2ba8220f81a4d6cb3611ca53db91fe1 P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:03.518435 12494 tablet_copy_client.cc:1002] T e2ba8220f81a4d6cb3611ca53db91fe1 P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Injecting latency into file download: 1500
I20240502 07:59:03.519110 9535 tablet_copy_service.cc:140] P a201db721bda4dc3a4422dc8fa9bf72e: Received BeginTabletCopySession request for tablet eddeb6a23ec1418eb35b414f5d52e7b7 from peer 8eb9945f989d4be6b51e1531a1365350 ({username='slave'} at 127.3.139.70:59727)
I20240502 07:59:03.519207 9535 tablet_copy_service.cc:161] P a201db721bda4dc3a4422dc8fa9bf72e: Beginning new tablet copy session on tablet eddeb6a23ec1418eb35b414f5d52e7b7 from peer 8eb9945f989d4be6b51e1531a1365350 at {username='slave'} at 127.3.139.70:59727: session id = 8eb9945f989d4be6b51e1531a1365350-eddeb6a23ec1418eb35b414f5d52e7b7
I20240502 07:59:03.520025 9535 tablet_copy_source_session.cc:205] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:03.520864 12497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eddeb6a23ec1418eb35b414f5d52e7b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:03.523227 12497 tablet_copy_client.cc:783] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:03.523375 12497 tablet_copy_client.cc:647] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:03.524288 12498 tablet_copy_client.cc:1002] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Injecting latency into file download: 1500
W20240502 07:59:03.528138 12386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:03.528321 12386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:59:03.528378 12386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:03.531433 12386 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:59:03.531489 12386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:59:03.531575 12386 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.73
I20240502 07:59:03.535043 12386 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:39259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.73:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/data/info.pb
--webserver_interface=127.3.139.73
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.3.139.126:46479
--never_fsync=true
--heap_profile_path=/tmp/kudu.12386
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.73
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:59:03.535897 12386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:59:03.536948 12386 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:59:03.544657 12512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:03.544996 12513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:03.545349 12515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:03.545782 12386 server_base.cc:1034] running on GCE node
I20240502 07:59:03.546154 12386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:59:03.546665 12386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:59:03.547828 12386 hybrid_clock.cc:648] HybridClock initialized: now 1714636743547789 us; error 51 us; skew 500 ppm
I20240502 07:59:03.549621 12386 webserver.cc:460] Webserver started at http://127.3.139.73:43895/ using document root <none> and password file <none>
I20240502 07:59:03.550117 12386 fs_manager.cc:359] Metadata directory not provided
I20240502 07:59:03.550168 12386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:59:03.550347 12386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:59:03.552037 12386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/data/instance:
uuid: "09ec2b380cbf4200af90b0eee891c010"
format_stamp: "Formatted at 2024-05-02 07:59:03 on dist-test-slave-twcr"
I20240502 07:59:03.552481 12386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/wal/instance:
uuid: "09ec2b380cbf4200af90b0eee891c010"
format_stamp: "Formatted at 2024-05-02 07:59:03 on dist-test-slave-twcr"
I20240502 07:59:03.555603 12386 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240502 07:59:03.557832 12521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:03.558771 12386 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240502 07:59:03.558894 12386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/wal
uuid: "09ec2b380cbf4200af90b0eee891c010"
format_stamp: "Formatted at 2024-05-02 07:59:03 on dist-test-slave-twcr"
I20240502 07:59:03.558991 12386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:59:03.573043 12386 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:59:03.573551 12386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:59:03.573683 12386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:59:03.574158 12386 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:59:03.574949 12386 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:59:03.574995 12386 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:03.575033 12386 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:59:03.575101 12386 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:03.583964 12386 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.73:35761
I20240502 07:59:03.584012 12634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.73:35761 every 8 connection(s)
I20240502 07:59:03.584862 12386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/data/info.pb
I20240502 07:59:03.593109 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 12386
I20240502 07:59:03.593142 12635 heartbeater.cc:347] Connected to a master server at 127.3.139.126:46479
I20240502 07:59:03.593217 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/wal/instance
I20240502 07:59:03.593331 12635 heartbeater.cc:464] Registering TS with master...
I20240502 07:59:03.593787 12635 heartbeater.cc:511] Master 127.3.139.126:46479 requested a full tablet report, sending...
I20240502 07:59:03.594645 9370 ts_manager.cc:194] Registered new tserver with Master: 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73:35761)
I20240502 07:59:03.595196 9370 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.73:54987
I20240502 07:59:04.596858 12635 heartbeater.cc:503] Master 127.3.139.126:46479 was elected leader, sending a full tablet report...
I20240502 07:59:04.607450 12198 tablet_copy_client.cc:523] T 92f37161960e4ee894c75d9034434001 P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:04.609427 12198 tablet_bootstrap.cc:492] T 92f37161960e4ee894c75d9034434001 P 8eb9945f989d4be6b51e1531a1365350: Bootstrap starting.
I20240502 07:59:04.614583 12198 log.cc:824] T 92f37161960e4ee894c75d9034434001 P 8eb9945f989d4be6b51e1531a1365350: Log is configured to *not* fsync() on all Append() calls
I20240502 07:59:04.633126 12198 tablet_bootstrap.cc:492] T 92f37161960e4ee894c75d9034434001 P 8eb9945f989d4be6b51e1531a1365350: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:04.633430 12198 tablet_bootstrap.cc:492] T 92f37161960e4ee894c75d9034434001 P 8eb9945f989d4be6b51e1531a1365350: Bootstrap complete.
I20240502 07:59:04.633700 12198 ts_tablet_manager.cc:1367] T 92f37161960e4ee894c75d9034434001 P 8eb9945f989d4be6b51e1531a1365350: Time spent bootstrapping tablet: real 0.024s user 0.023s sys 0.000s
I20240502 07:59:04.635622 12198 raft_consensus.cc:348] T 92f37161960e4ee894c75d9034434001 P 8eb9945f989d4be6b51e1531a1365350 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } }
I20240502 07:59:04.635836 12198 raft_consensus.cc:729] T 92f37161960e4ee894c75d9034434001 P 8eb9945f989d4be6b51e1531a1365350 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 8eb9945f989d4be6b51e1531a1365350, State: Initialized, Role: LEARNER
I20240502 07:59:04.636130 12198 consensus_queue.cc:260] T 92f37161960e4ee894c75d9034434001 P 8eb9945f989d4be6b51e1531a1365350 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } }
I20240502 07:59:04.636809 12198 ts_tablet_manager.cc:1398] T 92f37161960e4ee894c75d9034434001 P 8eb9945f989d4be6b51e1531a1365350: Time spent starting tablet: real 0.003s user 0.005s sys 0.000s
I20240502 07:59:04.637542 9797 tablet_copy_service.cc:342] P 1472c64688914149a318c7b8850a3df3: Request end of tablet copy session 8eb9945f989d4be6b51e1531a1365350-92f37161960e4ee894c75d9034434001 received from {username='slave'} at 127.3.139.70:41973
I20240502 07:59:04.637645 9797 tablet_copy_service.cc:433] P 1472c64688914149a318c7b8850a3df3: ending tablet copy session 8eb9945f989d4be6b51e1531a1365350-92f37161960e4ee894c75d9034434001 on tablet 92f37161960e4ee894c75d9034434001 with peer 8eb9945f989d4be6b51e1531a1365350
I20240502 07:59:04.639573 12231 tablet_copy_client.cc:523] T a12a084dba23425996e9122c083c1444 P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:04.641527 12231 tablet_bootstrap.cc:492] T a12a084dba23425996e9122c083c1444 P 51e82953ed2247fa9ff24638eab3508a: Bootstrap starting.
I20240502 07:59:04.647128 12231 log.cc:824] T a12a084dba23425996e9122c083c1444 P 51e82953ed2247fa9ff24638eab3508a: Log is configured to *not* fsync() on all Append() calls
I20240502 07:59:04.670778 12267 tablet_copy_client.cc:523] T 8ceb129e9cbe4c469e7dfba419719556 P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:04.670843 12231 tablet_bootstrap.cc:492] T a12a084dba23425996e9122c083c1444 P 51e82953ed2247fa9ff24638eab3508a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:04.671226 12231 tablet_bootstrap.cc:492] T a12a084dba23425996e9122c083c1444 P 51e82953ed2247fa9ff24638eab3508a: Bootstrap complete.
I20240502 07:59:04.671551 12231 ts_tablet_manager.cc:1367] T a12a084dba23425996e9122c083c1444 P 51e82953ed2247fa9ff24638eab3508a: Time spent bootstrapping tablet: real 0.030s user 0.021s sys 0.009s
I20240502 07:59:04.672246 12267 tablet_bootstrap.cc:492] T 8ceb129e9cbe4c469e7dfba419719556 P 51e82953ed2247fa9ff24638eab3508a: Bootstrap starting.
I20240502 07:59:04.674343 12231 raft_consensus.cc:348] T a12a084dba23425996e9122c083c1444 P 51e82953ed2247fa9ff24638eab3508a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } }
I20240502 07:59:04.674593 12231 raft_consensus.cc:729] T a12a084dba23425996e9122c083c1444 P 51e82953ed2247fa9ff24638eab3508a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 51e82953ed2247fa9ff24638eab3508a, State: Initialized, Role: LEARNER
I20240502 07:59:04.674928 12231 consensus_queue.cc:260] T a12a084dba23425996e9122c083c1444 P 51e82953ed2247fa9ff24638eab3508a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } }
I20240502 07:59:04.675868 12231 ts_tablet_manager.cc:1398] T a12a084dba23425996e9122c083c1444 P 51e82953ed2247fa9ff24638eab3508a: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20240502 07:59:04.676752 9535 tablet_copy_service.cc:342] P a201db721bda4dc3a4422dc8fa9bf72e: Request end of tablet copy session 51e82953ed2247fa9ff24638eab3508a-a12a084dba23425996e9122c083c1444 received from {username='slave'} at 127.3.139.69:55681
I20240502 07:59:04.676857 9535 tablet_copy_service.cc:433] P a201db721bda4dc3a4422dc8fa9bf72e: ending tablet copy session 51e82953ed2247fa9ff24638eab3508a-a12a084dba23425996e9122c083c1444 on tablet a12a084dba23425996e9122c083c1444 with peer 51e82953ed2247fa9ff24638eab3508a
I20240502 07:59:04.692876 12277 tablet_copy_client.cc:523] T af07c527fff94115a743df1c73c3f4b5 P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:04.693895 12267 tablet_bootstrap.cc:492] T 8ceb129e9cbe4c469e7dfba419719556 P 51e82953ed2247fa9ff24638eab3508a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:04.694113 12267 tablet_bootstrap.cc:492] T 8ceb129e9cbe4c469e7dfba419719556 P 51e82953ed2247fa9ff24638eab3508a: Bootstrap complete.
I20240502 07:59:04.694262 12267 ts_tablet_manager.cc:1367] T 8ceb129e9cbe4c469e7dfba419719556 P 51e82953ed2247fa9ff24638eab3508a: Time spent bootstrapping tablet: real 0.022s user 0.016s sys 0.004s
I20240502 07:59:04.694398 12277 tablet_bootstrap.cc:492] T af07c527fff94115a743df1c73c3f4b5 P 8eb9945f989d4be6b51e1531a1365350: Bootstrap starting.
I20240502 07:59:04.694818 12267 raft_consensus.cc:348] T 8ceb129e9cbe4c469e7dfba419719556 P 51e82953ed2247fa9ff24638eab3508a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } }
I20240502 07:59:04.694931 12267 raft_consensus.cc:729] T 8ceb129e9cbe4c469e7dfba419719556 P 51e82953ed2247fa9ff24638eab3508a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 51e82953ed2247fa9ff24638eab3508a, State: Initialized, Role: LEARNER
I20240502 07:59:04.695026 12267 consensus_queue.cc:260] T 8ceb129e9cbe4c469e7dfba419719556 P 51e82953ed2247fa9ff24638eab3508a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } }
I20240502 07:59:04.695514 11391 raft_consensus.cc:1203] T 92f37161960e4ee894c75d9034434001 P 8eb9945f989d4be6b51e1531a1365350 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:04.695586 12267 ts_tablet_manager.cc:1398] T 8ceb129e9cbe4c469e7dfba419719556 P 51e82953ed2247fa9ff24638eab3508a: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 07:59:04.695838 12285 tablet_copy_client.cc:523] T fd251de859e049c38a632a17b2fbdeae P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:04.697067 12285 tablet_bootstrap.cc:492] T fd251de859e049c38a632a17b2fbdeae P 8eb9945f989d4be6b51e1531a1365350: Bootstrap starting.
I20240502 07:59:04.699601 9666 tablet_copy_service.cc:342] P 8337af1943564c0dad18b45a597b805a: Request end of tablet copy session 51e82953ed2247fa9ff24638eab3508a-8ceb129e9cbe4c469e7dfba419719556 received from {username='slave'} at 127.3.139.69:35563
I20240502 07:59:04.699707 9666 tablet_copy_service.cc:433] P 8337af1943564c0dad18b45a597b805a: ending tablet copy session 51e82953ed2247fa9ff24638eab3508a-8ceb129e9cbe4c469e7dfba419719556 on tablet 8ceb129e9cbe4c469e7dfba419719556 with peer 51e82953ed2247fa9ff24638eab3508a
I20240502 07:59:04.718214 12277 tablet_bootstrap.cc:492] T af07c527fff94115a743df1c73c3f4b5 P 8eb9945f989d4be6b51e1531a1365350: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:04.718587 12277 tablet_bootstrap.cc:492] T af07c527fff94115a743df1c73c3f4b5 P 8eb9945f989d4be6b51e1531a1365350: Bootstrap complete.
I20240502 07:59:04.718799 12277 ts_tablet_manager.cc:1367] T af07c527fff94115a743df1c73c3f4b5 P 8eb9945f989d4be6b51e1531a1365350: Time spent bootstrapping tablet: real 0.024s user 0.014s sys 0.008s
I20240502 07:59:04.719353 12277 raft_consensus.cc:348] T af07c527fff94115a743df1c73c3f4b5 P 8eb9945f989d4be6b51e1531a1365350 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } }
I20240502 07:59:04.719517 12277 raft_consensus.cc:729] T af07c527fff94115a743df1c73c3f4b5 P 8eb9945f989d4be6b51e1531a1365350 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 8eb9945f989d4be6b51e1531a1365350, State: Initialized, Role: LEARNER
I20240502 07:59:04.719630 12277 consensus_queue.cc:260] T af07c527fff94115a743df1c73c3f4b5 P 8eb9945f989d4be6b51e1531a1365350 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } }
I20240502 07:59:04.719805 12285 tablet_bootstrap.cc:492] T fd251de859e049c38a632a17b2fbdeae P 8eb9945f989d4be6b51e1531a1365350: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:04.719895 12277 ts_tablet_manager.cc:1398] T af07c527fff94115a743df1c73c3f4b5 P 8eb9945f989d4be6b51e1531a1365350: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:04.720029 12285 tablet_bootstrap.cc:492] T fd251de859e049c38a632a17b2fbdeae P 8eb9945f989d4be6b51e1531a1365350: Bootstrap complete.
I20240502 07:59:04.720196 12285 ts_tablet_manager.cc:1367] T fd251de859e049c38a632a17b2fbdeae P 8eb9945f989d4be6b51e1531a1365350: Time spent bootstrapping tablet: real 0.023s user 0.021s sys 0.000s
I20240502 07:59:04.720594 9928 tablet_copy_service.cc:342] P 6cc5b9962ed24441bcba0c07b9f5438e: Request end of tablet copy session 8eb9945f989d4be6b51e1531a1365350-af07c527fff94115a743df1c73c3f4b5 received from {username='slave'} at 127.3.139.70:41665
I20240502 07:59:04.720687 9928 tablet_copy_service.cc:433] P 6cc5b9962ed24441bcba0c07b9f5438e: ending tablet copy session 8eb9945f989d4be6b51e1531a1365350-af07c527fff94115a743df1c73c3f4b5 on tablet af07c527fff94115a743df1c73c3f4b5 with peer 8eb9945f989d4be6b51e1531a1365350
I20240502 07:59:04.721045 12285 raft_consensus.cc:348] T fd251de859e049c38a632a17b2fbdeae P 8eb9945f989d4be6b51e1531a1365350 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } }
I20240502 07:59:04.721140 12285 raft_consensus.cc:729] T fd251de859e049c38a632a17b2fbdeae P 8eb9945f989d4be6b51e1531a1365350 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 8eb9945f989d4be6b51e1531a1365350, State: Initialized, Role: LEARNER
I20240502 07:59:04.721225 12285 consensus_queue.cc:260] T fd251de859e049c38a632a17b2fbdeae P 8eb9945f989d4be6b51e1531a1365350 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } }
I20240502 07:59:04.721554 12285 ts_tablet_manager.cc:1398] T fd251de859e049c38a632a17b2fbdeae P 8eb9945f989d4be6b51e1531a1365350: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:04.722086 9928 tablet_copy_service.cc:342] P 6cc5b9962ed24441bcba0c07b9f5438e: Request end of tablet copy session 8eb9945f989d4be6b51e1531a1365350-fd251de859e049c38a632a17b2fbdeae received from {username='slave'} at 127.3.139.70:41665
I20240502 07:59:04.722153 9928 tablet_copy_service.cc:433] P 6cc5b9962ed24441bcba0c07b9f5438e: ending tablet copy session 8eb9945f989d4be6b51e1531a1365350-fd251de859e049c38a632a17b2fbdeae on tablet fd251de859e049c38a632a17b2fbdeae with peer 8eb9945f989d4be6b51e1531a1365350
I20240502 07:59:04.728785 12297 tablet_copy_client.cc:523] T ecd0995c31fe4e708712c62ca60511c9 P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:04.730669 12297 tablet_bootstrap.cc:492] T ecd0995c31fe4e708712c62ca60511c9 P 98bef2e94ade4763a66c1ad68259744a: Bootstrap starting.
I20240502 07:59:04.736356 12297 log.cc:824] T ecd0995c31fe4e708712c62ca60511c9 P 98bef2e94ade4763a66c1ad68259744a: Log is configured to *not* fsync() on all Append() calls
I20240502 07:59:04.741761 12303 tablet_copy_client.cc:523] T 0fab37c27e184a2cacb08f05457fa6ae P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:04.743723 12303 tablet_bootstrap.cc:492] T 0fab37c27e184a2cacb08f05457fa6ae P a95f1d3af33f4c879fd290ca917d6329: Bootstrap starting.
I20240502 07:59:04.749923 12303 log.cc:824] T 0fab37c27e184a2cacb08f05457fa6ae P a95f1d3af33f4c879fd290ca917d6329: Log is configured to *not* fsync() on all Append() calls
I20240502 07:59:04.757982 12297 tablet_bootstrap.cc:492] T ecd0995c31fe4e708712c62ca60511c9 P 98bef2e94ade4763a66c1ad68259744a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:04.758307 12297 tablet_bootstrap.cc:492] T ecd0995c31fe4e708712c62ca60511c9 P 98bef2e94ade4763a66c1ad68259744a: Bootstrap complete.
I20240502 07:59:04.758595 12297 ts_tablet_manager.cc:1367] T ecd0995c31fe4e708712c62ca60511c9 P 98bef2e94ade4763a66c1ad68259744a: Time spent bootstrapping tablet: real 0.028s user 0.027s sys 0.000s
I20240502 07:59:04.760846 12297 raft_consensus.cc:348] T ecd0995c31fe4e708712c62ca60511c9 P 98bef2e94ade4763a66c1ad68259744a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } }
I20240502 07:59:04.761060 12297 raft_consensus.cc:729] T ecd0995c31fe4e708712c62ca60511c9 P 98bef2e94ade4763a66c1ad68259744a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 98bef2e94ade4763a66c1ad68259744a, State: Initialized, Role: LEARNER
I20240502 07:59:04.761399 12297 consensus_queue.cc:260] T ecd0995c31fe4e708712c62ca60511c9 P 98bef2e94ade4763a66c1ad68259744a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } }
I20240502 07:59:04.762199 12297 ts_tablet_manager.cc:1398] T ecd0995c31fe4e708712c62ca60511c9 P 98bef2e94ade4763a66c1ad68259744a: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20240502 07:59:04.762986 9666 tablet_copy_service.cc:342] P 8337af1943564c0dad18b45a597b805a: Request end of tablet copy session 98bef2e94ade4763a66c1ad68259744a-ecd0995c31fe4e708712c62ca60511c9 received from {username='slave'} at 127.3.139.71:52353
I20240502 07:59:04.763079 9666 tablet_copy_service.cc:433] P 8337af1943564c0dad18b45a597b805a: ending tablet copy session 98bef2e94ade4763a66c1ad68259744a-ecd0995c31fe4e708712c62ca60511c9 on tablet ecd0995c31fe4e708712c62ca60511c9 with peer 98bef2e94ade4763a66c1ad68259744a
I20240502 07:59:04.774705 12303 tablet_bootstrap.cc:492] T 0fab37c27e184a2cacb08f05457fa6ae P a95f1d3af33f4c879fd290ca917d6329: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:04.774971 12303 tablet_bootstrap.cc:492] T 0fab37c27e184a2cacb08f05457fa6ae P a95f1d3af33f4c879fd290ca917d6329: Bootstrap complete.
I20240502 07:59:04.775254 12303 ts_tablet_manager.cc:1367] T 0fab37c27e184a2cacb08f05457fa6ae P a95f1d3af33f4c879fd290ca917d6329: Time spent bootstrapping tablet: real 0.032s user 0.027s sys 0.003s
I20240502 07:59:04.777128 12303 raft_consensus.cc:348] T 0fab37c27e184a2cacb08f05457fa6ae P a95f1d3af33f4c879fd290ca917d6329 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } }
I20240502 07:59:04.777344 12303 raft_consensus.cc:729] T 0fab37c27e184a2cacb08f05457fa6ae P a95f1d3af33f4c879fd290ca917d6329 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a95f1d3af33f4c879fd290ca917d6329, State: Initialized, Role: LEARNER
I20240502 07:59:04.777640 12303 consensus_queue.cc:260] T 0fab37c27e184a2cacb08f05457fa6ae P a95f1d3af33f4c879fd290ca917d6329 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } }
I20240502 07:59:04.778424 12303 ts_tablet_manager.cc:1398] T 0fab37c27e184a2cacb08f05457fa6ae P a95f1d3af33f4c879fd290ca917d6329: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20240502 07:59:04.779013 9928 tablet_copy_service.cc:342] P 6cc5b9962ed24441bcba0c07b9f5438e: Request end of tablet copy session a95f1d3af33f4c879fd290ca917d6329-0fab37c27e184a2cacb08f05457fa6ae received from {username='slave'} at 127.3.139.72:51283
I20240502 07:59:04.779124 9928 tablet_copy_service.cc:433] P 6cc5b9962ed24441bcba0c07b9f5438e: ending tablet copy session a95f1d3af33f4c879fd290ca917d6329-0fab37c27e184a2cacb08f05457fa6ae on tablet 0fab37c27e184a2cacb08f05457fa6ae with peer a95f1d3af33f4c879fd290ca917d6329
I20240502 07:59:04.829334 11655 raft_consensus.cc:1203] T 0fab37c27e184a2cacb08f05457fa6ae P a95f1d3af33f4c879fd290ca917d6329 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:04.845973 12343 tablet_copy_client.cc:523] T 364d227472894712a2ae15f4589a5a5b P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:04.847508 12343 tablet_bootstrap.cc:492] T 364d227472894712a2ae15f4589a5a5b P 98bef2e94ade4763a66c1ad68259744a: Bootstrap starting.
I20240502 07:59:04.860805 12342 tablet_copy_client.cc:523] T d562a250cee247a48e573b25ecf00429 P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:04.862236 12342 tablet_bootstrap.cc:492] T d562a250cee247a48e573b25ecf00429 P a95f1d3af33f4c879fd290ca917d6329: Bootstrap starting.
I20240502 07:59:04.863866 12341 tablet_copy_client.cc:523] T 686b5cf6614d45c18cc776bd79567667 P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:04.865134 12341 tablet_bootstrap.cc:492] T 686b5cf6614d45c18cc776bd79567667 P a95f1d3af33f4c879fd290ca917d6329: Bootstrap starting.
I20240502 07:59:04.867647 12343 tablet_bootstrap.cc:492] T 364d227472894712a2ae15f4589a5a5b P 98bef2e94ade4763a66c1ad68259744a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:04.867910 12343 tablet_bootstrap.cc:492] T 364d227472894712a2ae15f4589a5a5b P 98bef2e94ade4763a66c1ad68259744a: Bootstrap complete.
I20240502 07:59:04.868081 12343 ts_tablet_manager.cc:1367] T 364d227472894712a2ae15f4589a5a5b P 98bef2e94ade4763a66c1ad68259744a: Time spent bootstrapping tablet: real 0.021s user 0.019s sys 0.001s
I20240502 07:59:04.868651 12343 raft_consensus.cc:348] T 364d227472894712a2ae15f4589a5a5b P 98bef2e94ade4763a66c1ad68259744a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } }
I20240502 07:59:04.868784 12343 raft_consensus.cc:729] T 364d227472894712a2ae15f4589a5a5b P 98bef2e94ade4763a66c1ad68259744a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 98bef2e94ade4763a66c1ad68259744a, State: Initialized, Role: LEARNER
I20240502 07:59:04.868881 12343 consensus_queue.cc:260] T 364d227472894712a2ae15f4589a5a5b P 98bef2e94ade4763a66c1ad68259744a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } }
I20240502 07:59:04.869132 12343 ts_tablet_manager.cc:1398] T 364d227472894712a2ae15f4589a5a5b P 98bef2e94ade4763a66c1ad68259744a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:04.869745 9666 tablet_copy_service.cc:342] P 8337af1943564c0dad18b45a597b805a: Request end of tablet copy session 98bef2e94ade4763a66c1ad68259744a-364d227472894712a2ae15f4589a5a5b received from {username='slave'} at 127.3.139.71:52353
I20240502 07:59:04.869837 9666 tablet_copy_service.cc:433] P 8337af1943564c0dad18b45a597b805a: ending tablet copy session 98bef2e94ade4763a66c1ad68259744a-364d227472894712a2ae15f4589a5a5b on tablet 364d227472894712a2ae15f4589a5a5b with peer 98bef2e94ade4763a66c1ad68259744a
I20240502 07:59:04.883008 12361 tablet_copy_client.cc:523] T d3db04c54a184eb8a1c23fec9e7145eb P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:04.884171 12342 tablet_bootstrap.cc:492] T d562a250cee247a48e573b25ecf00429 P a95f1d3af33f4c879fd290ca917d6329: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:04.884399 12342 tablet_bootstrap.cc:492] T d562a250cee247a48e573b25ecf00429 P a95f1d3af33f4c879fd290ca917d6329: Bootstrap complete.
I20240502 07:59:04.884549 12361 tablet_bootstrap.cc:492] T d3db04c54a184eb8a1c23fec9e7145eb P 51e82953ed2247fa9ff24638eab3508a: Bootstrap starting.
I20240502 07:59:04.884559 12342 ts_tablet_manager.cc:1367] T d562a250cee247a48e573b25ecf00429 P a95f1d3af33f4c879fd290ca917d6329: Time spent bootstrapping tablet: real 0.022s user 0.019s sys 0.002s
I20240502 07:59:04.884564 12341 tablet_bootstrap.cc:492] T 686b5cf6614d45c18cc776bd79567667 P a95f1d3af33f4c879fd290ca917d6329: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:04.884821 12341 tablet_bootstrap.cc:492] T 686b5cf6614d45c18cc776bd79567667 P a95f1d3af33f4c879fd290ca917d6329: Bootstrap complete.
I20240502 07:59:04.884972 12341 ts_tablet_manager.cc:1367] T 686b5cf6614d45c18cc776bd79567667 P a95f1d3af33f4c879fd290ca917d6329: Time spent bootstrapping tablet: real 0.020s user 0.019s sys 0.000s
I20240502 07:59:04.885095 12342 raft_consensus.cc:348] T d562a250cee247a48e573b25ecf00429 P a95f1d3af33f4c879fd290ca917d6329 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } }
I20240502 07:59:04.885200 12342 raft_consensus.cc:729] T d562a250cee247a48e573b25ecf00429 P a95f1d3af33f4c879fd290ca917d6329 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a95f1d3af33f4c879fd290ca917d6329, State: Initialized, Role: LEARNER
I20240502 07:59:04.885316 12342 consensus_queue.cc:260] T d562a250cee247a48e573b25ecf00429 P a95f1d3af33f4c879fd290ca917d6329 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } }
I20240502 07:59:04.885375 12341 raft_consensus.cc:348] T 686b5cf6614d45c18cc776bd79567667 P a95f1d3af33f4c879fd290ca917d6329 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } }
I20240502 07:59:04.885522 12341 raft_consensus.cc:729] T 686b5cf6614d45c18cc776bd79567667 P a95f1d3af33f4c879fd290ca917d6329 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a95f1d3af33f4c879fd290ca917d6329, State: Initialized, Role: LEARNER
I20240502 07:59:04.885561 12342 ts_tablet_manager.cc:1398] T d562a250cee247a48e573b25ecf00429 P a95f1d3af33f4c879fd290ca917d6329: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 07:59:04.885635 12341 consensus_queue.cc:260] T 686b5cf6614d45c18cc776bd79567667 P a95f1d3af33f4c879fd290ca917d6329 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } }
I20240502 07:59:04.885923 12341 ts_tablet_manager.cc:1398] T 686b5cf6614d45c18cc776bd79567667 P a95f1d3af33f4c879fd290ca917d6329: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:04.886407 9797 tablet_copy_service.cc:342] P 1472c64688914149a318c7b8850a3df3: Request end of tablet copy session a95f1d3af33f4c879fd290ca917d6329-686b5cf6614d45c18cc776bd79567667 received from {username='slave'} at 127.3.139.72:51819
I20240502 07:59:04.886497 9797 tablet_copy_service.cc:433] P 1472c64688914149a318c7b8850a3df3: ending tablet copy session a95f1d3af33f4c879fd290ca917d6329-686b5cf6614d45c18cc776bd79567667 on tablet 686b5cf6614d45c18cc776bd79567667 with peer a95f1d3af33f4c879fd290ca917d6329
I20240502 07:59:04.887220 12356 tablet_copy_client.cc:523] T 50f059668c6b4205a15d2c076e2789b0 P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:04.888077 9666 tablet_copy_service.cc:342] P 8337af1943564c0dad18b45a597b805a: Request end of tablet copy session a95f1d3af33f4c879fd290ca917d6329-d562a250cee247a48e573b25ecf00429 received from {username='slave'} at 127.3.139.72:42289
I20240502 07:59:04.888154 9666 tablet_copy_service.cc:433] P 8337af1943564c0dad18b45a597b805a: ending tablet copy session a95f1d3af33f4c879fd290ca917d6329-d562a250cee247a48e573b25ecf00429 on tablet d562a250cee247a48e573b25ecf00429 with peer a95f1d3af33f4c879fd290ca917d6329
I20240502 07:59:04.888635 12356 tablet_bootstrap.cc:492] T 50f059668c6b4205a15d2c076e2789b0 P 8eb9945f989d4be6b51e1531a1365350: Bootstrap starting.
I20240502 07:59:04.889289 12366 tablet_copy_client.cc:523] T fcdf47a04bdd4b9fa600a0bba59a4542 P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:04.890535 12366 tablet_bootstrap.cc:492] T fcdf47a04bdd4b9fa600a0bba59a4542 P 51e82953ed2247fa9ff24638eab3508a: Bootstrap starting.
I20240502 07:59:04.904217 12361 tablet_bootstrap.cc:492] T d3db04c54a184eb8a1c23fec9e7145eb P 51e82953ed2247fa9ff24638eab3508a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:04.904774 12361 tablet_bootstrap.cc:492] T d3db04c54a184eb8a1c23fec9e7145eb P 51e82953ed2247fa9ff24638eab3508a: Bootstrap complete.
I20240502 07:59:04.904932 12361 ts_tablet_manager.cc:1367] T d3db04c54a184eb8a1c23fec9e7145eb P 51e82953ed2247fa9ff24638eab3508a: Time spent bootstrapping tablet: real 0.020s user 0.018s sys 0.002s
I20240502 07:59:04.905483 12361 raft_consensus.cc:348] T d3db04c54a184eb8a1c23fec9e7145eb P 51e82953ed2247fa9ff24638eab3508a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } }
I20240502 07:59:04.905587 12361 raft_consensus.cc:729] T d3db04c54a184eb8a1c23fec9e7145eb P 51e82953ed2247fa9ff24638eab3508a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 51e82953ed2247fa9ff24638eab3508a, State: Initialized, Role: LEARNER
I20240502 07:59:04.905675 12361 consensus_queue.cc:260] T d3db04c54a184eb8a1c23fec9e7145eb P 51e82953ed2247fa9ff24638eab3508a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } }
I20240502 07:59:04.905968 12361 ts_tablet_manager.cc:1398] T d3db04c54a184eb8a1c23fec9e7145eb P 51e82953ed2247fa9ff24638eab3508a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:04.907311 9797 tablet_copy_service.cc:342] P 1472c64688914149a318c7b8850a3df3: Request end of tablet copy session 51e82953ed2247fa9ff24638eab3508a-d3db04c54a184eb8a1c23fec9e7145eb received from {username='slave'} at 127.3.139.69:36679
I20240502 07:59:04.907394 9797 tablet_copy_service.cc:433] P 1472c64688914149a318c7b8850a3df3: ending tablet copy session 51e82953ed2247fa9ff24638eab3508a-d3db04c54a184eb8a1c23fec9e7145eb on tablet d3db04c54a184eb8a1c23fec9e7145eb with peer 51e82953ed2247fa9ff24638eab3508a
I20240502 07:59:04.915520 12356 tablet_bootstrap.cc:492] T 50f059668c6b4205a15d2c076e2789b0 P 8eb9945f989d4be6b51e1531a1365350: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:04.915802 12356 tablet_bootstrap.cc:492] T 50f059668c6b4205a15d2c076e2789b0 P 8eb9945f989d4be6b51e1531a1365350: Bootstrap complete.
I20240502 07:59:04.915961 12356 ts_tablet_manager.cc:1367] T 50f059668c6b4205a15d2c076e2789b0 P 8eb9945f989d4be6b51e1531a1365350: Time spent bootstrapping tablet: real 0.027s user 0.019s sys 0.004s
I20240502 07:59:04.916363 12356 raft_consensus.cc:348] T 50f059668c6b4205a15d2c076e2789b0 P 8eb9945f989d4be6b51e1531a1365350 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } }
I20240502 07:59:04.916483 12356 raft_consensus.cc:729] T 50f059668c6b4205a15d2c076e2789b0 P 8eb9945f989d4be6b51e1531a1365350 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 8eb9945f989d4be6b51e1531a1365350, State: Initialized, Role: LEARNER
I20240502 07:59:04.916625 12356 consensus_queue.cc:260] T 50f059668c6b4205a15d2c076e2789b0 P 8eb9945f989d4be6b51e1531a1365350 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } }
I20240502 07:59:04.916901 12356 ts_tablet_manager.cc:1398] T 50f059668c6b4205a15d2c076e2789b0 P 8eb9945f989d4be6b51e1531a1365350: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:04.917495 9666 tablet_copy_service.cc:342] P 8337af1943564c0dad18b45a597b805a: Request end of tablet copy session 8eb9945f989d4be6b51e1531a1365350-50f059668c6b4205a15d2c076e2789b0 received from {username='slave'} at 127.3.139.70:48793
I20240502 07:59:04.917572 9666 tablet_copy_service.cc:433] P 8337af1943564c0dad18b45a597b805a: ending tablet copy session 8eb9945f989d4be6b51e1531a1365350-50f059668c6b4205a15d2c076e2789b0 on tablet 50f059668c6b4205a15d2c076e2789b0 with peer 8eb9945f989d4be6b51e1531a1365350
I20240502 07:59:04.919421 12366 tablet_bootstrap.cc:492] T fcdf47a04bdd4b9fa600a0bba59a4542 P 51e82953ed2247fa9ff24638eab3508a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:04.919651 12366 tablet_bootstrap.cc:492] T fcdf47a04bdd4b9fa600a0bba59a4542 P 51e82953ed2247fa9ff24638eab3508a: Bootstrap complete.
I20240502 07:59:04.919812 12366 ts_tablet_manager.cc:1367] T fcdf47a04bdd4b9fa600a0bba59a4542 P 51e82953ed2247fa9ff24638eab3508a: Time spent bootstrapping tablet: real 0.029s user 0.027s sys 0.000s
I20240502 07:59:04.920143 12366 raft_consensus.cc:348] T fcdf47a04bdd4b9fa600a0bba59a4542 P 51e82953ed2247fa9ff24638eab3508a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } }
I20240502 07:59:04.920254 12366 raft_consensus.cc:729] T fcdf47a04bdd4b9fa600a0bba59a4542 P 51e82953ed2247fa9ff24638eab3508a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 51e82953ed2247fa9ff24638eab3508a, State: Initialized, Role: LEARNER
I20240502 07:59:04.920346 12366 consensus_queue.cc:260] T fcdf47a04bdd4b9fa600a0bba59a4542 P 51e82953ed2247fa9ff24638eab3508a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } }
I20240502 07:59:04.920418 12397 tablet_copy_client.cc:523] T 3255d89d51a94426a78364230c9b249b P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:04.920608 12366 ts_tablet_manager.cc:1398] T fcdf47a04bdd4b9fa600a0bba59a4542 P 51e82953ed2247fa9ff24638eab3508a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:04.921761 12397 tablet_bootstrap.cc:492] T 3255d89d51a94426a78364230c9b249b P a95f1d3af33f4c879fd290ca917d6329: Bootstrap starting.
I20240502 07:59:04.923149 9666 tablet_copy_service.cc:342] P 8337af1943564c0dad18b45a597b805a: Request end of tablet copy session 51e82953ed2247fa9ff24638eab3508a-fcdf47a04bdd4b9fa600a0bba59a4542 received from {username='slave'} at 127.3.139.69:35563
I20240502 07:59:04.923230 9666 tablet_copy_service.cc:433] P 8337af1943564c0dad18b45a597b805a: ending tablet copy session 51e82953ed2247fa9ff24638eab3508a-fcdf47a04bdd4b9fa600a0bba59a4542 on tablet fcdf47a04bdd4b9fa600a0bba59a4542 with peer 51e82953ed2247fa9ff24638eab3508a
I20240502 07:59:04.937661 12417 tablet_copy_client.cc:523] T bc896a65284c48cd864bb6b57dbe5ca0 P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:04.938470 12397 tablet_bootstrap.cc:492] T 3255d89d51a94426a78364230c9b249b P a95f1d3af33f4c879fd290ca917d6329: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:04.938690 12397 tablet_bootstrap.cc:492] T 3255d89d51a94426a78364230c9b249b P a95f1d3af33f4c879fd290ca917d6329: Bootstrap complete.
I20240502 07:59:04.938838 12397 ts_tablet_manager.cc:1367] T 3255d89d51a94426a78364230c9b249b P a95f1d3af33f4c879fd290ca917d6329: Time spent bootstrapping tablet: real 0.017s user 0.013s sys 0.001s
I20240502 07:59:04.938946 12417 tablet_bootstrap.cc:492] T bc896a65284c48cd864bb6b57dbe5ca0 P a95f1d3af33f4c879fd290ca917d6329: Bootstrap starting.
I20240502 07:59:04.939253 12397 raft_consensus.cc:348] T 3255d89d51a94426a78364230c9b249b P a95f1d3af33f4c879fd290ca917d6329 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } }
I20240502 07:59:04.939360 12397 raft_consensus.cc:729] T 3255d89d51a94426a78364230c9b249b P a95f1d3af33f4c879fd290ca917d6329 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a95f1d3af33f4c879fd290ca917d6329, State: Initialized, Role: LEARNER
I20240502 07:59:04.939448 12397 consensus_queue.cc:260] T 3255d89d51a94426a78364230c9b249b P a95f1d3af33f4c879fd290ca917d6329 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } }
I20240502 07:59:04.939698 12397 ts_tablet_manager.cc:1398] T 3255d89d51a94426a78364230c9b249b P a95f1d3af33f4c879fd290ca917d6329: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240502 07:59:04.940279 9928 tablet_copy_service.cc:342] P 6cc5b9962ed24441bcba0c07b9f5438e: Request end of tablet copy session a95f1d3af33f4c879fd290ca917d6329-3255d89d51a94426a78364230c9b249b received from {username='slave'} at 127.3.139.72:51283
I20240502 07:59:04.940387 9928 tablet_copy_service.cc:433] P 6cc5b9962ed24441bcba0c07b9f5438e: ending tablet copy session a95f1d3af33f4c879fd290ca917d6329-3255d89d51a94426a78364230c9b249b on tablet 3255d89d51a94426a78364230c9b249b with peer a95f1d3af33f4c879fd290ca917d6329
I20240502 07:59:04.945489 12422 tablet_copy_client.cc:523] T 16c2ef3c0f5c44b39327175f6519fb95 P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:04.946945 12422 tablet_bootstrap.cc:492] T 16c2ef3c0f5c44b39327175f6519fb95 P 98bef2e94ade4763a66c1ad68259744a: Bootstrap starting.
I20240502 07:59:04.960659 12417 tablet_bootstrap.cc:492] T bc896a65284c48cd864bb6b57dbe5ca0 P a95f1d3af33f4c879fd290ca917d6329: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:04.960963 12417 tablet_bootstrap.cc:492] T bc896a65284c48cd864bb6b57dbe5ca0 P a95f1d3af33f4c879fd290ca917d6329: Bootstrap complete.
I20240502 07:59:04.961126 12417 ts_tablet_manager.cc:1367] T bc896a65284c48cd864bb6b57dbe5ca0 P a95f1d3af33f4c879fd290ca917d6329: Time spent bootstrapping tablet: real 0.022s user 0.021s sys 0.000s
I20240502 07:59:04.961542 12417 raft_consensus.cc:348] T bc896a65284c48cd864bb6b57dbe5ca0 P a95f1d3af33f4c879fd290ca917d6329 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } }
I20240502 07:59:04.961719 12417 raft_consensus.cc:729] T bc896a65284c48cd864bb6b57dbe5ca0 P a95f1d3af33f4c879fd290ca917d6329 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a95f1d3af33f4c879fd290ca917d6329, State: Initialized, Role: LEARNER
I20240502 07:59:04.961839 12417 consensus_queue.cc:260] T bc896a65284c48cd864bb6b57dbe5ca0 P a95f1d3af33f4c879fd290ca917d6329 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } }
I20240502 07:59:04.962098 12417 ts_tablet_manager.cc:1398] T bc896a65284c48cd864bb6b57dbe5ca0 P a95f1d3af33f4c879fd290ca917d6329: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240502 07:59:04.962615 9535 tablet_copy_service.cc:342] P a201db721bda4dc3a4422dc8fa9bf72e: Request end of tablet copy session a95f1d3af33f4c879fd290ca917d6329-bc896a65284c48cd864bb6b57dbe5ca0 received from {username='slave'} at 127.3.139.72:38421
I20240502 07:59:04.962702 9535 tablet_copy_service.cc:433] P a201db721bda4dc3a4422dc8fa9bf72e: ending tablet copy session a95f1d3af33f4c879fd290ca917d6329-bc896a65284c48cd864bb6b57dbe5ca0 on tablet bc896a65284c48cd864bb6b57dbe5ca0 with peer a95f1d3af33f4c879fd290ca917d6329
I20240502 07:59:04.967581 12442 tablet_copy_client.cc:523] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:04.968837 12422 tablet_bootstrap.cc:492] T 16c2ef3c0f5c44b39327175f6519fb95 P 98bef2e94ade4763a66c1ad68259744a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:04.968864 12442 tablet_bootstrap.cc:492] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 51e82953ed2247fa9ff24638eab3508a: Bootstrap starting.
I20240502 07:59:04.969103 12422 tablet_bootstrap.cc:492] T 16c2ef3c0f5c44b39327175f6519fb95 P 98bef2e94ade4763a66c1ad68259744a: Bootstrap complete.
I20240502 07:59:04.969254 12422 ts_tablet_manager.cc:1367] T 16c2ef3c0f5c44b39327175f6519fb95 P 98bef2e94ade4763a66c1ad68259744a: Time spent bootstrapping tablet: real 0.022s user 0.021s sys 0.000s
I20240502 07:59:04.969602 12422 raft_consensus.cc:348] T 16c2ef3c0f5c44b39327175f6519fb95 P 98bef2e94ade4763a66c1ad68259744a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } }
I20240502 07:59:04.969708 12422 raft_consensus.cc:729] T 16c2ef3c0f5c44b39327175f6519fb95 P 98bef2e94ade4763a66c1ad68259744a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 98bef2e94ade4763a66c1ad68259744a, State: Initialized, Role: LEARNER
I20240502 07:59:04.969791 12422 consensus_queue.cc:260] T 16c2ef3c0f5c44b39327175f6519fb95 P 98bef2e94ade4763a66c1ad68259744a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } }
I20240502 07:59:04.970073 12422 ts_tablet_manager.cc:1398] T 16c2ef3c0f5c44b39327175f6519fb95 P 98bef2e94ade4763a66c1ad68259744a: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240502 07:59:04.970650 9535 tablet_copy_service.cc:342] P a201db721bda4dc3a4422dc8fa9bf72e: Request end of tablet copy session 98bef2e94ade4763a66c1ad68259744a-16c2ef3c0f5c44b39327175f6519fb95 received from {username='slave'} at 127.3.139.71:43779
I20240502 07:59:04.970728 9535 tablet_copy_service.cc:433] P a201db721bda4dc3a4422dc8fa9bf72e: ending tablet copy session 98bef2e94ade4763a66c1ad68259744a-16c2ef3c0f5c44b39327175f6519fb95 on tablet 16c2ef3c0f5c44b39327175f6519fb95 with peer 98bef2e94ade4763a66c1ad68259744a
I20240502 07:59:04.986102 12442 tablet_bootstrap.cc:492] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 51e82953ed2247fa9ff24638eab3508a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:04.986341 12442 tablet_bootstrap.cc:492] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 51e82953ed2247fa9ff24638eab3508a: Bootstrap complete.
I20240502 07:59:04.986501 12442 ts_tablet_manager.cc:1367] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 51e82953ed2247fa9ff24638eab3508a: Time spent bootstrapping tablet: real 0.018s user 0.015s sys 0.000s
I20240502 07:59:04.986852 12442 raft_consensus.cc:348] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 51e82953ed2247fa9ff24638eab3508a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } }
I20240502 07:59:04.986986 12442 raft_consensus.cc:729] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 51e82953ed2247fa9ff24638eab3508a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 51e82953ed2247fa9ff24638eab3508a, State: Initialized, Role: LEARNER
I20240502 07:59:04.987100 12442 consensus_queue.cc:260] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 51e82953ed2247fa9ff24638eab3508a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } }
I20240502 07:59:04.987349 12442 ts_tablet_manager.cc:1398] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 51e82953ed2247fa9ff24638eab3508a: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240502 07:59:04.987900 9928 tablet_copy_service.cc:342] P 6cc5b9962ed24441bcba0c07b9f5438e: Request end of tablet copy session 51e82953ed2247fa9ff24638eab3508a-7da9d3b7fe8c40f6a68bb1b7a926d994 received from {username='slave'} at 127.3.139.69:48381
I20240502 07:59:04.987998 9928 tablet_copy_service.cc:433] P 6cc5b9962ed24441bcba0c07b9f5438e: ending tablet copy session 51e82953ed2247fa9ff24638eab3508a-7da9d3b7fe8c40f6a68bb1b7a926d994 on tablet 7da9d3b7fe8c40f6a68bb1b7a926d994 with peer 51e82953ed2247fa9ff24638eab3508a
I20240502 07:59:04.996953 11391 raft_consensus.cc:1203] T 50f059668c6b4205a15d2c076e2789b0 P 8eb9945f989d4be6b51e1531a1365350 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:05.007081 11391 raft_consensus.cc:1203] T fd251de859e049c38a632a17b2fbdeae P 8eb9945f989d4be6b51e1531a1365350 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:05.009147 12477 tablet_copy_client.cc:523] T 140868fcb19a4e7c837ce0c2c1c7041e P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:05.010557 12477 tablet_bootstrap.cc:492] T 140868fcb19a4e7c837ce0c2c1c7041e P 98bef2e94ade4763a66c1ad68259744a: Bootstrap starting.
I20240502 07:59:05.018694 12487 tablet_copy_client.cc:523] T e2ba8220f81a4d6cb3611ca53db91fe1 P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:05.020103 12487 tablet_bootstrap.cc:492] T e2ba8220f81a4d6cb3611ca53db91fe1 P 98bef2e94ade4763a66c1ad68259744a: Bootstrap starting.
I20240502 07:59:05.024545 12497 tablet_copy_client.cc:523] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:05.025729 12497 tablet_bootstrap.cc:492] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8eb9945f989d4be6b51e1531a1365350: Bootstrap starting.
I20240502 07:59:05.035383 12477 tablet_bootstrap.cc:492] T 140868fcb19a4e7c837ce0c2c1c7041e P 98bef2e94ade4763a66c1ad68259744a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:05.035704 12477 tablet_bootstrap.cc:492] T 140868fcb19a4e7c837ce0c2c1c7041e P 98bef2e94ade4763a66c1ad68259744a: Bootstrap complete.
I20240502 07:59:05.035905 12477 ts_tablet_manager.cc:1367] T 140868fcb19a4e7c837ce0c2c1c7041e P 98bef2e94ade4763a66c1ad68259744a: Time spent bootstrapping tablet: real 0.025s user 0.022s sys 0.003s
I20240502 07:59:05.036336 12477 raft_consensus.cc:348] T 140868fcb19a4e7c837ce0c2c1c7041e P 98bef2e94ade4763a66c1ad68259744a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } }
I20240502 07:59:05.036502 12477 raft_consensus.cc:729] T 140868fcb19a4e7c837ce0c2c1c7041e P 98bef2e94ade4763a66c1ad68259744a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 98bef2e94ade4763a66c1ad68259744a, State: Initialized, Role: LEARNER
I20240502 07:59:05.036617 12477 consensus_queue.cc:260] T 140868fcb19a4e7c837ce0c2c1c7041e P 98bef2e94ade4763a66c1ad68259744a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } }
I20240502 07:59:05.036989 12477 ts_tablet_manager.cc:1398] T 140868fcb19a4e7c837ce0c2c1c7041e P 98bef2e94ade4763a66c1ad68259744a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:05.037509 9535 tablet_copy_service.cc:342] P a201db721bda4dc3a4422dc8fa9bf72e: Request end of tablet copy session 98bef2e94ade4763a66c1ad68259744a-140868fcb19a4e7c837ce0c2c1c7041e received from {username='slave'} at 127.3.139.71:43779
I20240502 07:59:05.037600 9535 tablet_copy_service.cc:433] P a201db721bda4dc3a4422dc8fa9bf72e: ending tablet copy session 98bef2e94ade4763a66c1ad68259744a-140868fcb19a4e7c837ce0c2c1c7041e on tablet 140868fcb19a4e7c837ce0c2c1c7041e with peer 98bef2e94ade4763a66c1ad68259744a
I20240502 07:59:05.045513 11260 raft_consensus.cc:1203] T 8ceb129e9cbe4c469e7dfba419719556 P 51e82953ed2247fa9ff24638eab3508a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:05.046104 12487 tablet_bootstrap.cc:492] T e2ba8220f81a4d6cb3611ca53db91fe1 P 98bef2e94ade4763a66c1ad68259744a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:05.046525 12487 tablet_bootstrap.cc:492] T e2ba8220f81a4d6cb3611ca53db91fe1 P 98bef2e94ade4763a66c1ad68259744a: Bootstrap complete.
I20240502 07:59:05.046691 12487 ts_tablet_manager.cc:1367] T e2ba8220f81a4d6cb3611ca53db91fe1 P 98bef2e94ade4763a66c1ad68259744a: Time spent bootstrapping tablet: real 0.027s user 0.025s sys 0.000s
I20240502 07:59:05.047127 12487 raft_consensus.cc:348] T e2ba8220f81a4d6cb3611ca53db91fe1 P 98bef2e94ade4763a66c1ad68259744a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } }
I20240502 07:59:05.047336 12487 raft_consensus.cc:729] T e2ba8220f81a4d6cb3611ca53db91fe1 P 98bef2e94ade4763a66c1ad68259744a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 98bef2e94ade4763a66c1ad68259744a, State: Initialized, Role: LEARNER
I20240502 07:59:05.047448 12487 consensus_queue.cc:260] T e2ba8220f81a4d6cb3611ca53db91fe1 P 98bef2e94ade4763a66c1ad68259744a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } }
I20240502 07:59:05.047789 12487 ts_tablet_manager.cc:1398] T e2ba8220f81a4d6cb3611ca53db91fe1 P 98bef2e94ade4763a66c1ad68259744a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:05.048241 9535 tablet_copy_service.cc:342] P a201db721bda4dc3a4422dc8fa9bf72e: Request end of tablet copy session 98bef2e94ade4763a66c1ad68259744a-e2ba8220f81a4d6cb3611ca53db91fe1 received from {username='slave'} at 127.3.139.71:43779
I20240502 07:59:05.048327 9535 tablet_copy_service.cc:433] P a201db721bda4dc3a4422dc8fa9bf72e: ending tablet copy session 98bef2e94ade4763a66c1ad68259744a-e2ba8220f81a4d6cb3611ca53db91fe1 on tablet e2ba8220f81a4d6cb3611ca53db91fe1 with peer 98bef2e94ade4763a66c1ad68259744a
I20240502 07:59:05.057472 12497 tablet_bootstrap.cc:492] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8eb9945f989d4be6b51e1531a1365350: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:05.057765 12497 tablet_bootstrap.cc:492] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8eb9945f989d4be6b51e1531a1365350: Bootstrap complete.
I20240502 07:59:05.057938 12497 ts_tablet_manager.cc:1367] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8eb9945f989d4be6b51e1531a1365350: Time spent bootstrapping tablet: real 0.032s user 0.019s sys 0.000s
I20240502 07:59:05.058663 12497 raft_consensus.cc:348] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8eb9945f989d4be6b51e1531a1365350 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } }
I20240502 07:59:05.058790 12497 raft_consensus.cc:729] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8eb9945f989d4be6b51e1531a1365350 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 8eb9945f989d4be6b51e1531a1365350, State: Initialized, Role: LEARNER
I20240502 07:59:05.058930 12497 consensus_queue.cc:260] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8eb9945f989d4be6b51e1531a1365350 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } }
I20240502 07:59:05.059302 12497 ts_tablet_manager.cc:1398] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8eb9945f989d4be6b51e1531a1365350: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240502 07:59:05.059556 11260 raft_consensus.cc:1203] T fcdf47a04bdd4b9fa600a0bba59a4542 P 51e82953ed2247fa9ff24638eab3508a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:05.059835 9535 tablet_copy_service.cc:342] P a201db721bda4dc3a4422dc8fa9bf72e: Request end of tablet copy session 8eb9945f989d4be6b51e1531a1365350-eddeb6a23ec1418eb35b414f5d52e7b7 received from {username='slave'} at 127.3.139.70:59727
I20240502 07:59:05.059911 9535 tablet_copy_service.cc:433] P a201db721bda4dc3a4422dc8fa9bf72e: ending tablet copy session 8eb9945f989d4be6b51e1531a1365350-eddeb6a23ec1418eb35b414f5d52e7b7 on tablet eddeb6a23ec1418eb35b414f5d52e7b7 with peer 8eb9945f989d4be6b51e1531a1365350
I20240502 07:59:05.065593 11655 raft_consensus.cc:1203] T bc896a65284c48cd864bb6b57dbe5ca0 P a95f1d3af33f4c879fd290ca917d6329 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:05.071728 11522 raft_consensus.cc:1203] T 16c2ef3c0f5c44b39327175f6519fb95 P 98bef2e94ade4763a66c1ad68259744a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:05.102224 12643 raft_consensus.cc:1050] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3: attempting to promote NON_VOTER 8eb9945f989d4be6b51e1531a1365350 to VOTER
I20240502 07:59:05.102770 12643 consensus_queue.cc:237] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } }
I20240502 07:59:05.104152 11391 raft_consensus.cc:1261] T 92f37161960e4ee894c75d9034434001 P 8eb9945f989d4be6b51e1531a1365350 [term 1 LEARNER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.104405 9514 raft_consensus.cc:1261] T 92f37161960e4ee894c75d9034434001 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.104559 9908 raft_consensus.cc:1261] T 92f37161960e4ee894c75d9034434001 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.104804 13152 consensus_queue.cc:1035] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.105064 12643 consensus_queue.cc:1035] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.105382 12643 consensus_queue.cc:1035] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.107719 13153 raft_consensus.cc:2932] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:05.109063 11391 raft_consensus.cc:2932] T 92f37161960e4ee894c75d9034434001 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:05.109589 9514 raft_consensus.cc:2932] T 92f37161960e4ee894c75d9034434001 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:05.111194 9369 catalog_manager.cc:5484] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 reported cstate change: config changed from index 4 to 5, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1472c64688914149a318c7b8850a3df3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:05.112028 9908 raft_consensus.cc:2932] T 92f37161960e4ee894c75d9034434001 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:05.118052 9776 consensus_queue.cc:237] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } }
I20240502 07:59:05.118577 11391 raft_consensus.cc:1203] T af07c527fff94115a743df1c73c3f4b5 P 8eb9945f989d4be6b51e1531a1365350 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:05.119216 9908 raft_consensus.cc:1261] T 92f37161960e4ee894c75d9034434001 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:05.119216 11391 raft_consensus.cc:1261] T 92f37161960e4ee894c75d9034434001 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:05.119647 13156 consensus_queue.cc:1035] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:05.119828 13152 consensus_queue.cc:1035] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:05.121343 11260 raft_consensus.cc:1203] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 51e82953ed2247fa9ff24638eab3508a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:05.121385 11259 raft_consensus.cc:1203] T a12a084dba23425996e9122c083c1444 P 51e82953ed2247fa9ff24638eab3508a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:05.121356 13152 raft_consensus.cc:2932] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:05.122843 9360 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 92f37161960e4ee894c75d9034434001 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:05.124048 11391 raft_consensus.cc:2932] T 92f37161960e4ee894c75d9034434001 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:05.124279 9908 raft_consensus.cc:2932] T 92f37161960e4ee894c75d9034434001 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:05.126130 9365 catalog_manager.cc:5484] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 reported cstate change: config changed from index 5 to 6, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New cstate: current_term: 1 leader_uuid: "1472c64688914149a318c7b8850a3df3" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:05.127871 11522 raft_consensus.cc:1203] T 140868fcb19a4e7c837ce0c2c1c7041e P 98bef2e94ade4763a66c1ad68259744a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:05.128890 9493 tablet_service.cc:1514] Processing DeleteTablet for tablet 92f37161960e4ee894c75d9034434001 with delete_type TABLET_DATA_TOMBSTONED (TS a201db721bda4dc3a4422dc8fa9bf72e not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:41730
I20240502 07:59:05.129868 13175 tablet_replica.cc:330] T 92f37161960e4ee894c75d9034434001 P a201db721bda4dc3a4422dc8fa9bf72e: stopping tablet replica
I20240502 07:59:05.130525 13175 raft_consensus.cc:2229] T 92f37161960e4ee894c75d9034434001 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:05.130750 13175 raft_consensus.cc:2258] T 92f37161960e4ee894c75d9034434001 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:05.132118 13175 ts_tablet_manager.cc:1875] T 92f37161960e4ee894c75d9034434001 P a201db721bda4dc3a4422dc8fa9bf72e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:05.134203 13175 ts_tablet_manager.cc:1888] T 92f37161960e4ee894c75d9034434001 P a201db721bda4dc3a4422dc8fa9bf72e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 07:59:05.134291 13175 log.cc:1196] T 92f37161960e4ee894c75d9034434001 P a201db721bda4dc3a4422dc8fa9bf72e: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal/wals/92f37161960e4ee894c75d9034434001
I20240502 07:59:05.134845 9360 catalog_manager.cc:4830] TS a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551): tablet 92f37161960e4ee894c75d9034434001 (table rebalance_test_table_0 [id=efdfc231272e43a5afefacf0ec75ec57]) successfully deleted
I20240502 07:59:05.135164 11391 raft_consensus.cc:1203] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8eb9945f989d4be6b51e1531a1365350 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:05.148592 11655 raft_consensus.cc:1203] T 3255d89d51a94426a78364230c9b249b P a95f1d3af33f4c879fd290ca917d6329 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:05.173954 11655 raft_consensus.cc:1203] T 686b5cf6614d45c18cc776bd79567667 P a95f1d3af33f4c879fd290ca917d6329 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:05.191388 11522 raft_consensus.cc:1203] T 364d227472894712a2ae15f4589a5a5b P 98bef2e94ade4763a66c1ad68259744a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:05.213709 13182 raft_consensus.cc:1050] T 0fab37c27e184a2cacb08f05457fa6ae P 6cc5b9962ed24441bcba0c07b9f5438e: attempting to promote NON_VOTER a95f1d3af33f4c879fd290ca917d6329 to VOTER
I20240502 07:59:05.214298 13182 consensus_queue.cc:237] T 0fab37c27e184a2cacb08f05457fa6ae P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } }
I20240502 07:59:05.215646 9646 raft_consensus.cc:1261] T 0fab37c27e184a2cacb08f05457fa6ae P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.215673 11655 raft_consensus.cc:1261] T 0fab37c27e184a2cacb08f05457fa6ae P a95f1d3af33f4c879fd290ca917d6329 [term 1 LEARNER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.216100 13184 consensus_queue.cc:1035] T 0fab37c27e184a2cacb08f05457fa6ae P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.216284 9514 raft_consensus.cc:1261] T 0fab37c27e184a2cacb08f05457fa6ae P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.216311 13182 consensus_queue.cc:1035] T 0fab37c27e184a2cacb08f05457fa6ae P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.216663 12639 consensus_queue.cc:1035] T 0fab37c27e184a2cacb08f05457fa6ae P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.219087 13184 raft_consensus.cc:2932] T 0fab37c27e184a2cacb08f05457fa6ae P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:05.219792 11522 raft_consensus.cc:1203] T ecd0995c31fe4e708712c62ca60511c9 P 98bef2e94ade4763a66c1ad68259744a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:05.219893 9514 raft_consensus.cc:2932] T 0fab37c27e184a2cacb08f05457fa6ae P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:05.219980 11655 raft_consensus.cc:2932] T 0fab37c27e184a2cacb08f05457fa6ae P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:05.222931 9367 catalog_manager.cc:5484] T 0fab37c27e184a2cacb08f05457fa6ae P a201db721bda4dc3a4422dc8fa9bf72e reported cstate change: config changed from index 4 to 5, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:05.223636 9646 raft_consensus.cc:2932] T 0fab37c27e184a2cacb08f05457fa6ae P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:05.226382 9906 consensus_queue.cc:237] T 0fab37c27e184a2cacb08f05457fa6ae P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } }
I20240502 07:59:05.227411 11655 raft_consensus.cc:1261] T 0fab37c27e184a2cacb08f05457fa6ae P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:05.227463 9514 raft_consensus.cc:1261] T 0fab37c27e184a2cacb08f05457fa6ae P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:05.227763 13181 consensus_queue.cc:1035] T 0fab37c27e184a2cacb08f05457fa6ae P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:05.227952 13182 consensus_queue.cc:1035] T 0fab37c27e184a2cacb08f05457fa6ae P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:05.229310 12639 raft_consensus.cc:2932] T 0fab37c27e184a2cacb08f05457fa6ae P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:05.229718 9514 raft_consensus.cc:2932] T 0fab37c27e184a2cacb08f05457fa6ae P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:05.229724 11655 raft_consensus.cc:2932] T 0fab37c27e184a2cacb08f05457fa6ae P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:05.230571 9358 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 0fab37c27e184a2cacb08f05457fa6ae with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:05.231503 9370 catalog_manager.cc:5484] T 0fab37c27e184a2cacb08f05457fa6ae P 6cc5b9962ed24441bcba0c07b9f5438e reported cstate change: config changed from index 5 to 6, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New cstate: current_term: 1 leader_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:05.233675 9624 tablet_service.cc:1514] Processing DeleteTablet for tablet 0fab37c27e184a2cacb08f05457fa6ae with delete_type TABLET_DATA_TOMBSTONED (TS 8337af1943564c0dad18b45a597b805a not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:53862
I20240502 07:59:05.234156 13188 tablet_replica.cc:330] T 0fab37c27e184a2cacb08f05457fa6ae P 8337af1943564c0dad18b45a597b805a: stopping tablet replica
I20240502 07:59:05.234437 13188 raft_consensus.cc:2229] T 0fab37c27e184a2cacb08f05457fa6ae P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:05.234632 13188 raft_consensus.cc:2258] T 0fab37c27e184a2cacb08f05457fa6ae P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:05.235808 13188 ts_tablet_manager.cc:1875] T 0fab37c27e184a2cacb08f05457fa6ae P 8337af1943564c0dad18b45a597b805a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:05.237509 13188 ts_tablet_manager.cc:1888] T 0fab37c27e184a2cacb08f05457fa6ae P 8337af1943564c0dad18b45a597b805a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 07:59:05.237586 13188 log.cc:1196] T 0fab37c27e184a2cacb08f05457fa6ae P 8337af1943564c0dad18b45a597b805a: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal/wals/0fab37c27e184a2cacb08f05457fa6ae
I20240502 07:59:05.238075 9358 catalog_manager.cc:4830] TS 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667): tablet 0fab37c27e184a2cacb08f05457fa6ae (table rebalance_test_table_9 [id=f0ea2346b9b9482e8343d199275d5819]) successfully deleted
I20240502 07:59:05.323915 11655 raft_consensus.cc:1203] T d562a250cee247a48e573b25ecf00429 P a95f1d3af33f4c879fd290ca917d6329 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:05.415211 12639 raft_consensus.cc:1050] T fd251de859e049c38a632a17b2fbdeae P 6cc5b9962ed24441bcba0c07b9f5438e: attempting to promote NON_VOTER 8eb9945f989d4be6b51e1531a1365350 to VOTER
I20240502 07:59:05.415634 12639 consensus_queue.cc:237] T fd251de859e049c38a632a17b2fbdeae P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } }
I20240502 07:59:05.416926 9514 raft_consensus.cc:1261] T fd251de859e049c38a632a17b2fbdeae P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.416936 9646 raft_consensus.cc:1261] T fd251de859e049c38a632a17b2fbdeae P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.416993 11391 raft_consensus.cc:1261] T fd251de859e049c38a632a17b2fbdeae P 8eb9945f989d4be6b51e1531a1365350 [term 1 LEARNER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.417358 13182 consensus_queue.cc:1035] T fd251de859e049c38a632a17b2fbdeae P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.417541 13184 consensus_queue.cc:1035] T fd251de859e049c38a632a17b2fbdeae P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.417855 12639 consensus_queue.cc:1035] T fd251de859e049c38a632a17b2fbdeae P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.420145 13182 raft_consensus.cc:2932] T fd251de859e049c38a632a17b2fbdeae P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:05.421818 9514 raft_consensus.cc:2932] T fd251de859e049c38a632a17b2fbdeae P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:05.422667 11391 raft_consensus.cc:2932] T fd251de859e049c38a632a17b2fbdeae P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:05.422132 9646 raft_consensus.cc:2932] T fd251de859e049c38a632a17b2fbdeae P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:05.423231 9365 catalog_manager.cc:5484] T fd251de859e049c38a632a17b2fbdeae P 6cc5b9962ed24441bcba0c07b9f5438e reported cstate change: config changed from index 4 to 5, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:05.426595 9906 consensus_queue.cc:237] T fd251de859e049c38a632a17b2fbdeae P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } }
I20240502 07:59:05.428283 11259 raft_consensus.cc:1203] T d3db04c54a184eb8a1c23fec9e7145eb P 51e82953ed2247fa9ff24638eab3508a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:05.428414 9646 raft_consensus.cc:1261] T fd251de859e049c38a632a17b2fbdeae P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:05.428946 13184 consensus_queue.cc:1035] T fd251de859e049c38a632a17b2fbdeae P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:05.430344 12639 raft_consensus.cc:2932] T fd251de859e049c38a632a17b2fbdeae P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:05.430627 11391 raft_consensus.cc:1261] T fd251de859e049c38a632a17b2fbdeae P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:05.430692 9644 raft_consensus.cc:2932] T fd251de859e049c38a632a17b2fbdeae P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:05.432382 9358 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet fd251de859e049c38a632a17b2fbdeae with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:05.433022 13181 consensus_queue.cc:1035] T fd251de859e049c38a632a17b2fbdeae P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:05.433324 9370 catalog_manager.cc:5484] T fd251de859e049c38a632a17b2fbdeae P 8337af1943564c0dad18b45a597b805a reported cstate change: config changed from index 5 to 6, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New cstate: current_term: 1 leader_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:05.434274 11391 raft_consensus.cc:2932] T fd251de859e049c38a632a17b2fbdeae P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:05.435739 9493 tablet_service.cc:1514] Processing DeleteTablet for tablet fd251de859e049c38a632a17b2fbdeae with delete_type TABLET_DATA_TOMBSTONED (TS a201db721bda4dc3a4422dc8fa9bf72e not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:41730
I20240502 07:59:05.435937 13175 tablet_replica.cc:330] T fd251de859e049c38a632a17b2fbdeae P a201db721bda4dc3a4422dc8fa9bf72e: stopping tablet replica
I20240502 07:59:05.436077 13175 raft_consensus.cc:2229] T fd251de859e049c38a632a17b2fbdeae P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:05.436211 13175 raft_consensus.cc:2258] T fd251de859e049c38a632a17b2fbdeae P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:05.436973 13175 ts_tablet_manager.cc:1875] T fd251de859e049c38a632a17b2fbdeae P a201db721bda4dc3a4422dc8fa9bf72e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:05.438935 13175 ts_tablet_manager.cc:1888] T fd251de859e049c38a632a17b2fbdeae P a201db721bda4dc3a4422dc8fa9bf72e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 07:59:05.439004 13175 log.cc:1196] T fd251de859e049c38a632a17b2fbdeae P a201db721bda4dc3a4422dc8fa9bf72e: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal/wals/fd251de859e049c38a632a17b2fbdeae
I20240502 07:59:05.440079 9360 catalog_manager.cc:4830] TS a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551): tablet fd251de859e049c38a632a17b2fbdeae (table rebalance_test_table_8 [id=6aecdf32373e4c9db62c5a4844dc38ca]) successfully deleted
I20240502 07:59:05.461503 9644 consensus_queue.cc:237] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } }
I20240502 07:59:05.462728 9906 raft_consensus.cc:1261] T 427c037fa09c4b94b7bc6c71bdb61c2f P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:05.462836 9776 raft_consensus.cc:1261] T 427c037fa09c4b94b7bc6c71bdb61c2f P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 07:59:05.463133 9580 consensus_peers.cc:485] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a -> Peer 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70:45329): Couldn't send request to peer 8eb9945f989d4be6b51e1531a1365350. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 427c037fa09c4b94b7bc6c71bdb61c2f. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:05.463299 13299 consensus_queue.cc:1035] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:05.463488 12641 consensus_queue.cc:1035] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:05.465126 13299 raft_consensus.cc:2932] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } } }
I20240502 07:59:05.465565 9906 raft_consensus.cc:2932] T 427c037fa09c4b94b7bc6c71bdb61c2f P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } } }
I20240502 07:59:05.466001 9776 raft_consensus.cc:2932] T 427c037fa09c4b94b7bc6c71bdb61c2f P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } } }
I20240502 07:59:05.467793 9370 catalog_manager.cc:5484] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a reported cstate change: config changed from index -1 to 4, NON_VOTER 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) added. New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:05.489354 12641 raft_consensus.cc:1050] T fcdf47a04bdd4b9fa600a0bba59a4542 P 8337af1943564c0dad18b45a597b805a: attempting to promote NON_VOTER 51e82953ed2247fa9ff24638eab3508a to VOTER
I20240502 07:59:05.489806 12641 consensus_queue.cc:237] T fcdf47a04bdd4b9fa600a0bba59a4542 P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } }
I20240502 07:59:05.491176 9906 raft_consensus.cc:1261] T fcdf47a04bdd4b9fa600a0bba59a4542 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.491180 9514 raft_consensus.cc:1261] T fcdf47a04bdd4b9fa600a0bba59a4542 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.491804 12084 consensus_queue.cc:1035] T fcdf47a04bdd4b9fa600a0bba59a4542 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.491887 11259 raft_consensus.cc:1261] T fcdf47a04bdd4b9fa600a0bba59a4542 P 51e82953ed2247fa9ff24638eab3508a [term 1 LEARNER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.492295 13328 consensus_queue.cc:1035] T fcdf47a04bdd4b9fa600a0bba59a4542 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.492484 12641 consensus_queue.cc:1035] T fcdf47a04bdd4b9fa600a0bba59a4542 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.494354 12641 raft_consensus.cc:2932] T fcdf47a04bdd4b9fa600a0bba59a4542 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:05.495867 11259 raft_consensus.cc:2932] T fcdf47a04bdd4b9fa600a0bba59a4542 P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:05.496052 9906 raft_consensus.cc:2932] T fcdf47a04bdd4b9fa600a0bba59a4542 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:05.496423 9514 raft_consensus.cc:2932] T fcdf47a04bdd4b9fa600a0bba59a4542 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:05.498472 9369 catalog_manager.cc:5484] T fcdf47a04bdd4b9fa600a0bba59a4542 P 51e82953ed2247fa9ff24638eab3508a reported cstate change: config changed from index 4 to 5, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:05.501406 9644 consensus_queue.cc:237] T fcdf47a04bdd4b9fa600a0bba59a4542 P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } }
I20240502 07:59:05.502353 11259 raft_consensus.cc:1261] T fcdf47a04bdd4b9fa600a0bba59a4542 P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:05.502420 9514 raft_consensus.cc:1261] T fcdf47a04bdd4b9fa600a0bba59a4542 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:05.502714 13328 consensus_queue.cc:1035] T fcdf47a04bdd4b9fa600a0bba59a4542 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:05.503286 12641 consensus_queue.cc:1035] T fcdf47a04bdd4b9fa600a0bba59a4542 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:05.505278 12641 raft_consensus.cc:2932] T fcdf47a04bdd4b9fa600a0bba59a4542 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:05.505457 11572 raft_consensus.cc:1050] T 16c2ef3c0f5c44b39327175f6519fb95 P a201db721bda4dc3a4422dc8fa9bf72e: attempting to promote NON_VOTER 98bef2e94ade4763a66c1ad68259744a to VOTER
I20240502 07:59:05.505856 11572 consensus_queue.cc:237] T 16c2ef3c0f5c44b39327175f6519fb95 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } }
I20240502 07:59:05.506930 9358 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet fcdf47a04bdd4b9fa600a0bba59a4542 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:05.506978 11259 raft_consensus.cc:2932] T fcdf47a04bdd4b9fa600a0bba59a4542 P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:05.507275 9776 raft_consensus.cc:1261] T 16c2ef3c0f5c44b39327175f6519fb95 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.507447 11522 raft_consensus.cc:1261] T 16c2ef3c0f5c44b39327175f6519fb95 P 98bef2e94ade4763a66c1ad68259744a [term 1 LEARNER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.507956 11572 consensus_queue.cc:1035] T 16c2ef3c0f5c44b39327175f6519fb95 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.508250 11572 consensus_queue.cc:1035] T 16c2ef3c0f5c44b39327175f6519fb95 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.508881 9906 raft_consensus.cc:1261] T 16c2ef3c0f5c44b39327175f6519fb95 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.509264 9365 catalog_manager.cc:5484] T fcdf47a04bdd4b9fa600a0bba59a4542 P 51e82953ed2247fa9ff24638eab3508a reported cstate change: config changed from index 5 to 6, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:05.509415 13338 consensus_queue.cc:1035] T 16c2ef3c0f5c44b39327175f6519fb95 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.509379 9514 raft_consensus.cc:2932] T fcdf47a04bdd4b9fa600a0bba59a4542 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:05.511602 11521 raft_consensus.cc:1203] T e2ba8220f81a4d6cb3611ca53db91fe1 P 98bef2e94ade4763a66c1ad68259744a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:05.513208 9886 tablet_service.cc:1514] Processing DeleteTablet for tablet fcdf47a04bdd4b9fa600a0bba59a4542 with delete_type TABLET_DATA_TOMBSTONED (TS 6cc5b9962ed24441bcba0c07b9f5438e not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:41940
I20240502 07:59:05.514371 13349 tablet_replica.cc:330] T fcdf47a04bdd4b9fa600a0bba59a4542 P 6cc5b9962ed24441bcba0c07b9f5438e: stopping tablet replica
I20240502 07:59:05.514707 13349 raft_consensus.cc:2229] T fcdf47a04bdd4b9fa600a0bba59a4542 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:05.514938 13349 raft_consensus.cc:2258] T fcdf47a04bdd4b9fa600a0bba59a4542 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:05.515975 13337 raft_consensus.cc:2932] T 16c2ef3c0f5c44b39327175f6519fb95 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:05.516386 13349 ts_tablet_manager.cc:1875] T fcdf47a04bdd4b9fa600a0bba59a4542 P 6cc5b9962ed24441bcba0c07b9f5438e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:05.516439 9776 raft_consensus.cc:2932] T 16c2ef3c0f5c44b39327175f6519fb95 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:05.518599 13349 ts_tablet_manager.cc:1888] T fcdf47a04bdd4b9fa600a0bba59a4542 P 6cc5b9962ed24441bcba0c07b9f5438e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 07:59:05.518339 11522 raft_consensus.cc:2932] T 16c2ef3c0f5c44b39327175f6519fb95 P 98bef2e94ade4763a66c1ad68259744a [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:05.518683 13349 log.cc:1196] T fcdf47a04bdd4b9fa600a0bba59a4542 P 6cc5b9962ed24441bcba0c07b9f5438e: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal/wals/fcdf47a04bdd4b9fa600a0bba59a4542
I20240502 07:59:05.519109 9906 raft_consensus.cc:2932] T 16c2ef3c0f5c44b39327175f6519fb95 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:05.519316 9358 catalog_manager.cc:4830] TS 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379): tablet fcdf47a04bdd4b9fa600a0bba59a4542 (table rebalance_test_table_0 [id=efdfc231272e43a5afefacf0ec75ec57]) successfully deleted
I20240502 07:59:05.519606 9370 catalog_manager.cc:5484] T 16c2ef3c0f5c44b39327175f6519fb95 P 1472c64688914149a318c7b8850a3df3 reported cstate change: config changed from index 4 to 5, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:05.524464 9514 consensus_queue.cc:237] T 16c2ef3c0f5c44b39327175f6519fb95 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } }
I20240502 07:59:05.525493 9776 raft_consensus.cc:1261] T 16c2ef3c0f5c44b39327175f6519fb95 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:05.525489 11522 raft_consensus.cc:1261] T 16c2ef3c0f5c44b39327175f6519fb95 P 98bef2e94ade4763a66c1ad68259744a [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:05.525813 13337 consensus_queue.cc:1035] T 16c2ef3c0f5c44b39327175f6519fb95 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:05.525992 11572 consensus_queue.cc:1035] T 16c2ef3c0f5c44b39327175f6519fb95 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:05.527628 11572 raft_consensus.cc:2932] T 16c2ef3c0f5c44b39327175f6519fb95 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:05.528393 13184 raft_consensus.cc:1050] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e: attempting to promote NON_VOTER a95f1d3af33f4c879fd290ca917d6329 to VOTER
I20240502 07:59:05.528690 13184 consensus_queue.cc:237] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } }
I20240502 07:59:05.529428 9360 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 16c2ef3c0f5c44b39327175f6519fb95 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:05.529809 9777 raft_consensus.cc:2932] T 16c2ef3c0f5c44b39327175f6519fb95 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:05.530031 9644 raft_consensus.cc:1261] T 3255d89d51a94426a78364230c9b249b P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.530109 11655 raft_consensus.cc:1261] T 3255d89d51a94426a78364230c9b249b P a95f1d3af33f4c879fd290ca917d6329 [term 1 LEARNER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.530087 11522 raft_consensus.cc:2932] T 16c2ef3c0f5c44b39327175f6519fb95 P 98bef2e94ade4763a66c1ad68259744a [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:05.530366 13181 consensus_queue.cc:1035] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.530539 12639 consensus_queue.cc:1035] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.532341 9514 raft_consensus.cc:1261] T 3255d89d51a94426a78364230c9b249b P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.532261 9369 catalog_manager.cc:5484] T 16c2ef3c0f5c44b39327175f6519fb95 P 1472c64688914149a318c7b8850a3df3 reported cstate change: config changed from index 5 to 6, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:05.533015 12639 consensus_queue.cc:1035] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.534822 9886 tablet_service.cc:1514] Processing DeleteTablet for tablet 16c2ef3c0f5c44b39327175f6519fb95 with delete_type TABLET_DATA_TOMBSTONED (TS 6cc5b9962ed24441bcba0c07b9f5438e not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:41940
I20240502 07:59:05.534989 13349 tablet_replica.cc:330] T 16c2ef3c0f5c44b39327175f6519fb95 P 6cc5b9962ed24441bcba0c07b9f5438e: stopping tablet replica
I20240502 07:59:05.535121 13349 raft_consensus.cc:2229] T 16c2ef3c0f5c44b39327175f6519fb95 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:05.535240 13349 raft_consensus.cc:2258] T 16c2ef3c0f5c44b39327175f6519fb95 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:05.535539 13181 raft_consensus.cc:2932] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:05.535970 9644 raft_consensus.cc:2932] T 3255d89d51a94426a78364230c9b249b P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:05.536099 13349 ts_tablet_manager.cc:1875] T 16c2ef3c0f5c44b39327175f6519fb95 P 6cc5b9962ed24441bcba0c07b9f5438e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:05.537865 13349 ts_tablet_manager.cc:1888] T 16c2ef3c0f5c44b39327175f6519fb95 P 6cc5b9962ed24441bcba0c07b9f5438e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 07:59:05.537758 11655 raft_consensus.cc:2932] T 3255d89d51a94426a78364230c9b249b P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:05.537928 13349 log.cc:1196] T 16c2ef3c0f5c44b39327175f6519fb95 P 6cc5b9962ed24441bcba0c07b9f5438e: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal/wals/16c2ef3c0f5c44b39327175f6519fb95
I20240502 07:59:05.538125 9365 catalog_manager.cc:5484] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e reported cstate change: config changed from index 4 to 5, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:05.538940 9514 raft_consensus.cc:2932] T 3255d89d51a94426a78364230c9b249b P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:05.539134 9358 catalog_manager.cc:4830] TS 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379): tablet 16c2ef3c0f5c44b39327175f6519fb95 (table rebalance_test_table_2 [id=1b8f60b832d543568624eb22602bb97c]) successfully deleted
I20240502 07:59:05.541990 9906 consensus_queue.cc:237] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } }
I20240502 07:59:05.543434 11655 raft_consensus.cc:1261] T 3255d89d51a94426a78364230c9b249b P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:05.544121 9644 raft_consensus.cc:1261] T 3255d89d51a94426a78364230c9b249b P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:05.544746 13184 consensus_queue.cc:1035] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:05.545051 13184 consensus_queue.cc:1035] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:05.546486 12639 raft_consensus.cc:2932] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:05.546940 11655 raft_consensus.cc:2932] T 3255d89d51a94426a78364230c9b249b P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:05.547200 9644 raft_consensus.cc:2932] T 3255d89d51a94426a78364230c9b249b P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:05.548174 9358 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 3255d89d51a94426a78364230c9b249b with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:05.549351 9367 catalog_manager.cc:5484] T 3255d89d51a94426a78364230c9b249b P a95f1d3af33f4c879fd290ca917d6329 reported cstate change: config changed from index 5 to 6, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New cstate: current_term: 1 leader_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:05.551599 9493 tablet_service.cc:1514] Processing DeleteTablet for tablet 3255d89d51a94426a78364230c9b249b with delete_type TABLET_DATA_TOMBSTONED (TS a201db721bda4dc3a4422dc8fa9bf72e not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:41730
I20240502 07:59:05.551797 13175 tablet_replica.cc:330] T 3255d89d51a94426a78364230c9b249b P a201db721bda4dc3a4422dc8fa9bf72e: stopping tablet replica
I20240502 07:59:05.551941 13175 raft_consensus.cc:2229] T 3255d89d51a94426a78364230c9b249b P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:05.552083 13175 raft_consensus.cc:2258] T 3255d89d51a94426a78364230c9b249b P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:05.552644 13175 ts_tablet_manager.cc:1875] T 3255d89d51a94426a78364230c9b249b P a201db721bda4dc3a4422dc8fa9bf72e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:05.554389 13175 ts_tablet_manager.cc:1888] T 3255d89d51a94426a78364230c9b249b P a201db721bda4dc3a4422dc8fa9bf72e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 07:59:05.554435 13175 log.cc:1196] T 3255d89d51a94426a78364230c9b249b P a201db721bda4dc3a4422dc8fa9bf72e: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal/wals/3255d89d51a94426a78364230c9b249b
I20240502 07:59:05.554873 9360 catalog_manager.cc:4830] TS a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551): tablet 3255d89d51a94426a78364230c9b249b (table rebalance_test_table_2 [id=1b8f60b832d543568624eb22602bb97c]) successfully deleted
I20240502 07:59:05.558411 13338 raft_consensus.cc:1050] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e: attempting to promote NON_VOTER 51e82953ed2247fa9ff24638eab3508a to VOTER
I20240502 07:59:05.558781 13338 consensus_queue.cc:237] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } }
I20240502 07:59:05.560061 9906 raft_consensus.cc:1261] T a12a084dba23425996e9122c083c1444 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.560075 11259 raft_consensus.cc:1261] T a12a084dba23425996e9122c083c1444 P 51e82953ed2247fa9ff24638eab3508a [term 1 LEARNER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.560144 9776 raft_consensus.cc:1261] T a12a084dba23425996e9122c083c1444 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.560468 13337 consensus_queue.cc:1035] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.560642 13339 consensus_queue.cc:1035] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.560963 13337 consensus_queue.cc:1035] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.562551 13339 raft_consensus.cc:2932] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:05.563813 11259 raft_consensus.cc:2932] T a12a084dba23425996e9122c083c1444 P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:05.564055 9776 raft_consensus.cc:2932] T a12a084dba23425996e9122c083c1444 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:05.566758 9369 catalog_manager.cc:5484] T a12a084dba23425996e9122c083c1444 P 51e82953ed2247fa9ff24638eab3508a reported cstate change: config changed from index 4 to 5, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:05.567641 11572 raft_consensus.cc:1050] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e: attempting to promote NON_VOTER 8eb9945f989d4be6b51e1531a1365350 to VOTER
I20240502 07:59:05.567997 11572 consensus_queue.cc:237] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } }
I20240502 07:59:05.569890 9906 raft_consensus.cc:1261] T eddeb6a23ec1418eb35b414f5d52e7b7 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.569882 11391 raft_consensus.cc:1261] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8eb9945f989d4be6b51e1531a1365350 [term 1 LEARNER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.569939 9644 raft_consensus.cc:1261] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.570369 11572 consensus_queue.cc:1035] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.570295 9906 raft_consensus.cc:2932] T a12a084dba23425996e9122c083c1444 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:05.570578 13338 consensus_queue.cc:1035] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.570684 13339 consensus_queue.cc:1035] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.573899 9514 consensus_queue.cc:237] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } }
I20240502 07:59:05.574419 13383 raft_consensus.cc:2932] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:05.575112 11259 raft_consensus.cc:1261] T a12a084dba23425996e9122c083c1444 P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:05.575508 13384 consensus_queue.cc:1035] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:05.575731 9644 raft_consensus.cc:2932] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:05.576489 9906 raft_consensus.cc:1261] T a12a084dba23425996e9122c083c1444 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:05.577028 11391 raft_consensus.cc:2932] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:05.577162 13390 consensus_queue.cc:1035] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:05.577458 9369 catalog_manager.cc:5484] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e reported cstate change: config changed from index 4 to 5, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:05.578461 9906 raft_consensus.cc:2932] T eddeb6a23ec1418eb35b414f5d52e7b7 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:05.579314 13384 raft_consensus.cc:2932] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:05.581393 9514 consensus_queue.cc:237] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } }
I20240502 07:59:05.582158 9360 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet a12a084dba23425996e9122c083c1444 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:05.582311 11391 raft_consensus.cc:1261] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:05.582336 9644 raft_consensus.cc:1261] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:05.582698 13337 consensus_queue.cc:1035] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:05.582862 13383 consensus_queue.cc:1035] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:05.583557 11259 raft_consensus.cc:2932] T a12a084dba23425996e9122c083c1444 P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:05.583990 9370 catalog_manager.cc:5484] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e reported cstate change: config changed from index 5 to 6, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:05.584637 13299 raft_consensus.cc:1050] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a: attempting to promote NON_VOTER 51e82953ed2247fa9ff24638eab3508a to VOTER
I20240502 07:59:05.584944 13299 consensus_queue.cc:237] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } }
I20240502 07:59:05.586122 9906 raft_consensus.cc:2932] T a12a084dba23425996e9122c083c1444 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:05.586346 11259 raft_consensus.cc:1261] T 8ceb129e9cbe4c469e7dfba419719556 P 51e82953ed2247fa9ff24638eab3508a [term 1 LEARNER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.586442 9776 raft_consensus.cc:1261] T 8ceb129e9cbe4c469e7dfba419719556 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.586851 12084 consensus_queue.cc:1035] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.587208 12084 consensus_queue.cc:1035] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.588589 9908 raft_consensus.cc:1261] T 8ceb129e9cbe4c469e7dfba419719556 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.588938 12641 consensus_queue.cc:1035] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.590389 13383 raft_consensus.cc:2932] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:05.590745 9644 raft_consensus.cc:2932] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:05.591570 9360 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet eddeb6a23ec1418eb35b414f5d52e7b7 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:05.592206 9757 tablet_service.cc:1514] Processing DeleteTablet for tablet a12a084dba23425996e9122c083c1444 with delete_type TABLET_DATA_TOMBSTONED (TS 1472c64688914149a318c7b8850a3df3 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:57656
I20240502 07:59:05.593008 9367 catalog_manager.cc:5484] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8337af1943564c0dad18b45a597b805a reported cstate change: config changed from index 5 to 6, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:05.593472 11391 raft_consensus.cc:2932] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:05.594538 12641 raft_consensus.cc:2932] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:05.595840 13404 tablet_replica.cc:330] T a12a084dba23425996e9122c083c1444 P 1472c64688914149a318c7b8850a3df3: stopping tablet replica
I20240502 07:59:05.596215 13404 raft_consensus.cc:2229] T a12a084dba23425996e9122c083c1444 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:05.596446 13404 raft_consensus.cc:2258] T a12a084dba23425996e9122c083c1444 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:05.596951 9776 raft_consensus.cc:2932] T 8ceb129e9cbe4c469e7dfba419719556 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:05.597605 13404 ts_tablet_manager.cc:1875] T a12a084dba23425996e9122c083c1444 P 1472c64688914149a318c7b8850a3df3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:05.599526 9886 tablet_service.cc:1514] Processing DeleteTablet for tablet eddeb6a23ec1418eb35b414f5d52e7b7 with delete_type TABLET_DATA_TOMBSTONED (TS 6cc5b9962ed24441bcba0c07b9f5438e not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:41940
I20240502 07:59:05.599720 13349 tablet_replica.cc:330] T eddeb6a23ec1418eb35b414f5d52e7b7 P 6cc5b9962ed24441bcba0c07b9f5438e: stopping tablet replica
I20240502 07:59:05.599694 9908 raft_consensus.cc:2932] T 8ceb129e9cbe4c469e7dfba419719556 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:05.599846 13349 raft_consensus.cc:2229] T eddeb6a23ec1418eb35b414f5d52e7b7 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:05.599967 13349 raft_consensus.cc:2258] T eddeb6a23ec1418eb35b414f5d52e7b7 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:05.600850 13404 ts_tablet_manager.cc:1888] T a12a084dba23425996e9122c083c1444 P 1472c64688914149a318c7b8850a3df3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 07:59:05.600970 13404 log.cc:1196] T a12a084dba23425996e9122c083c1444 P 1472c64688914149a318c7b8850a3df3: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal/wals/a12a084dba23425996e9122c083c1444
I20240502 07:59:05.601569 9367 catalog_manager.cc:5484] T 8ceb129e9cbe4c469e7dfba419719556 P 1472c64688914149a318c7b8850a3df3 reported cstate change: config changed from index 4 to 5, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:05.602059 9360 catalog_manager.cc:4830] TS 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191): tablet a12a084dba23425996e9122c083c1444 (table rebalance_test_table_3 [id=47c27ad65ee1405898b667c2642e1b3a]) successfully deleted
I20240502 07:59:05.602960 13349 ts_tablet_manager.cc:1875] T eddeb6a23ec1418eb35b414f5d52e7b7 P 6cc5b9962ed24441bcba0c07b9f5438e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:05.604789 13349 ts_tablet_manager.cc:1888] T eddeb6a23ec1418eb35b414f5d52e7b7 P 6cc5b9962ed24441bcba0c07b9f5438e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 07:59:05.604840 13349 log.cc:1196] T eddeb6a23ec1418eb35b414f5d52e7b7 P 6cc5b9962ed24441bcba0c07b9f5438e: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal/wals/eddeb6a23ec1418eb35b414f5d52e7b7
I20240502 07:59:05.606073 9358 catalog_manager.cc:4830] TS 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379): tablet eddeb6a23ec1418eb35b414f5d52e7b7 (table rebalance_test_table_2 [id=1b8f60b832d543568624eb22602bb97c]) successfully deleted
I20240502 07:59:05.608094 12641 raft_consensus.cc:1050] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a: attempting to promote NON_VOTER 8eb9945f989d4be6b51e1531a1365350 to VOTER
I20240502 07:59:05.608453 12641 consensus_queue.cc:237] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } }
I20240502 07:59:05.609753 9776 raft_consensus.cc:1261] T 50f059668c6b4205a15d2c076e2789b0 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.610080 13299 consensus_queue.cc:1035] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.610136 11391 raft_consensus.cc:1261] T 50f059668c6b4205a15d2c076e2789b0 P 8eb9945f989d4be6b51e1531a1365350 [term 1 LEARNER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.610970 9908 raft_consensus.cc:1261] T 50f059668c6b4205a15d2c076e2789b0 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.610939 13299 consensus_queue.cc:1035] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.611552 13328 consensus_queue.cc:1035] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.613240 12084 raft_consensus.cc:1050] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a: attempting to promote NON_VOTER 98bef2e94ade4763a66c1ad68259744a to VOTER
I20240502 07:59:05.613598 12084 consensus_queue.cc:237] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } }
I20240502 07:59:05.614877 9777 raft_consensus.cc:1261] T 364d227472894712a2ae15f4589a5a5b P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.615000 11522 raft_consensus.cc:1261] T 364d227472894712a2ae15f4589a5a5b P 98bef2e94ade4763a66c1ad68259744a [term 1 LEARNER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.615278 9514 raft_consensus.cc:1261] T 364d227472894712a2ae15f4589a5a5b P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.615651 13415 consensus_queue.cc:1035] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.615860 12641 consensus_queue.cc:1035] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.616197 12641 consensus_queue.cc:1035] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.617766 13328 raft_consensus.cc:2932] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:05.619509 11522 raft_consensus.cc:2932] T 364d227472894712a2ae15f4589a5a5b P 98bef2e94ade4763a66c1ad68259744a [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:05.619769 9514 raft_consensus.cc:2932] T 364d227472894712a2ae15f4589a5a5b P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:05.620031 9370 catalog_manager.cc:5484] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a reported cstate change: config changed from index 4 to 5, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:05.622963 9777 raft_consensus.cc:2932] T 364d227472894712a2ae15f4589a5a5b P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:05.624176 9644 consensus_queue.cc:237] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } }
I20240502 07:59:05.626164 11522 raft_consensus.cc:1261] T 364d227472894712a2ae15f4589a5a5b P 98bef2e94ade4763a66c1ad68259744a [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:05.626505 12084 consensus_queue.cc:1035] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:05.627158 11259 raft_consensus.cc:2932] T 8ceb129e9cbe4c469e7dfba419719556 P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:05.627415 9514 raft_consensus.cc:1261] T 364d227472894712a2ae15f4589a5a5b P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:05.627748 12084 consensus_queue.cc:1035] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:05.627763 12641 raft_consensus.cc:2932] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:05.630149 12084 raft_consensus.cc:2932] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:05.630604 9514 raft_consensus.cc:2932] T 364d227472894712a2ae15f4589a5a5b P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:05.631495 9908 raft_consensus.cc:2932] T 50f059668c6b4205a15d2c076e2789b0 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:05.632861 9776 raft_consensus.cc:2932] T 50f059668c6b4205a15d2c076e2789b0 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:05.633713 9370 catalog_manager.cc:5484] T 50f059668c6b4205a15d2c076e2789b0 P 6cc5b9962ed24441bcba0c07b9f5438e reported cstate change: config changed from index 4 to 5, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:05.634825 9644 consensus_queue.cc:237] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } }
I20240502 07:59:05.635655 11522 raft_consensus.cc:2932] T 364d227472894712a2ae15f4589a5a5b P 98bef2e94ade4763a66c1ad68259744a [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:05.636277 11259 raft_consensus.cc:1261] T 8ceb129e9cbe4c469e7dfba419719556 P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:05.636749 9776 raft_consensus.cc:1261] T 8ceb129e9cbe4c469e7dfba419719556 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:05.637092 13299 consensus_queue.cc:1035] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:05.637327 13328 consensus_queue.cc:1035] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:05.637840 9365 catalog_manager.cc:5484] T 364d227472894712a2ae15f4589a5a5b P 98bef2e94ade4763a66c1ad68259744a reported cstate change: config changed from index 5 to 6, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:05.638356 9358 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 364d227472894712a2ae15f4589a5a5b with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:05.638831 13328 raft_consensus.cc:2932] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:05.639860 9757 tablet_service.cc:1514] Processing DeleteTablet for tablet 364d227472894712a2ae15f4589a5a5b with delete_type TABLET_DATA_TOMBSTONED (TS 1472c64688914149a318c7b8850a3df3 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:57656
I20240502 07:59:05.640061 13404 tablet_replica.cc:330] T 364d227472894712a2ae15f4589a5a5b P 1472c64688914149a318c7b8850a3df3: stopping tablet replica
I20240502 07:59:05.640225 13404 raft_consensus.cc:2229] T 364d227472894712a2ae15f4589a5a5b P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:05.640370 13404 raft_consensus.cc:2258] T 364d227472894712a2ae15f4589a5a5b P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:05.640547 9358 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 8ceb129e9cbe4c469e7dfba419719556 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:05.640698 11259 raft_consensus.cc:2932] T 8ceb129e9cbe4c469e7dfba419719556 P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:05.641013 9776 raft_consensus.cc:2932] T 8ceb129e9cbe4c469e7dfba419719556 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:05.641681 9370 catalog_manager.cc:5484] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a reported cstate change: config changed from index 5 to 6, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:05.641849 13404 ts_tablet_manager.cc:1875] T 364d227472894712a2ae15f4589a5a5b P 1472c64688914149a318c7b8850a3df3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:05.643782 13404 ts_tablet_manager.cc:1888] T 364d227472894712a2ae15f4589a5a5b P 1472c64688914149a318c7b8850a3df3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 07:59:05.643858 13404 log.cc:1196] T 364d227472894712a2ae15f4589a5a5b P 1472c64688914149a318c7b8850a3df3: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal/wals/364d227472894712a2ae15f4589a5a5b
I20240502 07:59:05.643949 9644 consensus_queue.cc:237] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } }
I20240502 07:59:05.644349 9360 catalog_manager.cc:4830] TS 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191): tablet 364d227472894712a2ae15f4589a5a5b (table rebalance_test_table_6 [id=af6c4a01744d4fcfbdf53cfc8d19500b]) successfully deleted
I20240502 07:59:05.645107 11391 raft_consensus.cc:2932] T 50f059668c6b4205a15d2c076e2789b0 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:05.646590 9908 raft_consensus.cc:1261] T 50f059668c6b4205a15d2c076e2789b0 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:05.646914 11389 raft_consensus.cc:1261] T 50f059668c6b4205a15d2c076e2789b0 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:05.646970 13421 consensus_queue.cc:1035] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:05.647776 9886 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ceb129e9cbe4c469e7dfba419719556 with delete_type TABLET_DATA_TOMBSTONED (TS 6cc5b9962ed24441bcba0c07b9f5438e not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:41940
I20240502 07:59:05.647971 13349 tablet_replica.cc:330] T 8ceb129e9cbe4c469e7dfba419719556 P 6cc5b9962ed24441bcba0c07b9f5438e: stopping tablet replica
I20240502 07:59:05.648089 13349 raft_consensus.cc:2229] T 8ceb129e9cbe4c469e7dfba419719556 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:05.648209 13349 raft_consensus.cc:2258] T 8ceb129e9cbe4c469e7dfba419719556 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:05.648468 13420 consensus_queue.cc:1035] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:05.648775 13349 ts_tablet_manager.cc:1875] T 8ceb129e9cbe4c469e7dfba419719556 P 6cc5b9962ed24441bcba0c07b9f5438e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:05.650697 13349 ts_tablet_manager.cc:1888] T 8ceb129e9cbe4c469e7dfba419719556 P 6cc5b9962ed24441bcba0c07b9f5438e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 07:59:05.650758 13349 log.cc:1196] T 8ceb129e9cbe4c469e7dfba419719556 P 6cc5b9962ed24441bcba0c07b9f5438e: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal/wals/8ceb129e9cbe4c469e7dfba419719556
I20240502 07:59:05.651207 9358 catalog_manager.cc:4830] TS 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379): tablet 8ceb129e9cbe4c469e7dfba419719556 (table rebalance_test_table_9 [id=f0ea2346b9b9482e8343d199275d5819]) successfully deleted
I20240502 07:59:05.651607 13415 raft_consensus.cc:2932] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:05.652957 11389 raft_consensus.cc:2932] T 50f059668c6b4205a15d2c076e2789b0 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:05.653231 9908 raft_consensus.cc:2932] T 50f059668c6b4205a15d2c076e2789b0 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:05.655210 9369 catalog_manager.cc:5484] T 50f059668c6b4205a15d2c076e2789b0 P 8eb9945f989d4be6b51e1531a1365350 reported cstate change: config changed from index 5 to 6, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:05.655426 9358 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 50f059668c6b4205a15d2c076e2789b0 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:05.657625 9757 tablet_service.cc:1514] Processing DeleteTablet for tablet 50f059668c6b4205a15d2c076e2789b0 with delete_type TABLET_DATA_TOMBSTONED (TS 1472c64688914149a318c7b8850a3df3 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:57656
I20240502 07:59:05.657842 13404 tablet_replica.cc:330] T 50f059668c6b4205a15d2c076e2789b0 P 1472c64688914149a318c7b8850a3df3: stopping tablet replica
I20240502 07:59:05.657974 13404 raft_consensus.cc:2229] T 50f059668c6b4205a15d2c076e2789b0 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:05.658104 13404 raft_consensus.cc:2258] T 50f059668c6b4205a15d2c076e2789b0 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:05.658847 13404 ts_tablet_manager.cc:1875] T 50f059668c6b4205a15d2c076e2789b0 P 1472c64688914149a318c7b8850a3df3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:05.660574 13404 ts_tablet_manager.cc:1888] T 50f059668c6b4205a15d2c076e2789b0 P 1472c64688914149a318c7b8850a3df3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 07:59:05.660640 13404 log.cc:1196] T 50f059668c6b4205a15d2c076e2789b0 P 1472c64688914149a318c7b8850a3df3: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal/wals/50f059668c6b4205a15d2c076e2789b0
I20240502 07:59:05.661021 9360 catalog_manager.cc:4830] TS 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191): tablet 50f059668c6b4205a15d2c076e2789b0 (table rebalance_test_table_1 [id=982b41384c8f44b99bbe8eecffb1cd87]) successfully deleted
I20240502 07:59:05.668498 13439 raft_consensus.cc:1050] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3: attempting to promote NON_VOTER a95f1d3af33f4c879fd290ca917d6329 to VOTER
I20240502 07:59:05.668902 13439 consensus_queue.cc:237] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } }
I20240502 07:59:05.670136 11655 raft_consensus.cc:1261] T 686b5cf6614d45c18cc776bd79567667 P a95f1d3af33f4c879fd290ca917d6329 [term 1 LEARNER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.670217 9644 raft_consensus.cc:1261] T 686b5cf6614d45c18cc776bd79567667 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.670217 9514 raft_consensus.cc:1261] T 686b5cf6614d45c18cc776bd79567667 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.670701 13445 consensus_queue.cc:1035] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.670861 13152 consensus_queue.cc:1035] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.671043 13444 consensus_queue.cc:1035] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.673219 13445 raft_consensus.cc:2932] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:05.674052 9646 raft_consensus.cc:2932] T 686b5cf6614d45c18cc776bd79567667 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:05.675197 11655 raft_consensus.cc:2932] T 686b5cf6614d45c18cc776bd79567667 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:05.676046 9365 catalog_manager.cc:5484] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 reported cstate change: config changed from index 4 to 5, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1472c64688914149a318c7b8850a3df3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:05.676108 9514 raft_consensus.cc:2932] T 686b5cf6614d45c18cc776bd79567667 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:05.679028 9776 consensus_queue.cc:237] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } }
I20240502 07:59:05.680183 11655 raft_consensus.cc:1261] T 686b5cf6614d45c18cc776bd79567667 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:05.680197 9514 raft_consensus.cc:1261] T 686b5cf6614d45c18cc776bd79567667 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:05.680514 13444 consensus_queue.cc:1035] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:05.680663 13439 consensus_queue.cc:1035] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:05.682293 13445 raft_consensus.cc:2932] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:05.682965 13337 raft_consensus.cc:1050] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e: attempting to promote NON_VOTER a95f1d3af33f4c879fd290ca917d6329 to VOTER
I20240502 07:59:05.682947 9514 raft_consensus.cc:2932] T 686b5cf6614d45c18cc776bd79567667 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:05.683351 13337 consensus_queue.cc:237] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } }
I20240502 07:59:05.684376 9360 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 686b5cf6614d45c18cc776bd79567667 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:05.684885 11654 raft_consensus.cc:1261] T bc896a65284c48cd864bb6b57dbe5ca0 P a95f1d3af33f4c879fd290ca917d6329 [term 1 LEARNER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.684922 9369 catalog_manager.cc:5484] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 reported cstate change: config changed from index 5 to 6, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New cstate: current_term: 1 leader_uuid: "1472c64688914149a318c7b8850a3df3" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:05.685412 9777 raft_consensus.cc:1261] T bc896a65284c48cd864bb6b57dbe5ca0 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.685446 9646 raft_consensus.cc:1261] T bc896a65284c48cd864bb6b57dbe5ca0 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.685875 11572 consensus_queue.cc:1035] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.686183 11572 consensus_queue.cc:1035] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.686648 13339 consensus_queue.cc:1035] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.687732 9776 consensus_queue.cc:237] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } }
I20240502 07:59:05.687927 9624 tablet_service.cc:1514] Processing DeleteTablet for tablet 686b5cf6614d45c18cc776bd79567667 with delete_type TABLET_DATA_TOMBSTONED (TS 8337af1943564c0dad18b45a597b805a not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:53862
I20240502 07:59:05.688304 13188 tablet_replica.cc:330] T 686b5cf6614d45c18cc776bd79567667 P 8337af1943564c0dad18b45a597b805a: stopping tablet replica
I20240502 07:59:05.688419 13188 raft_consensus.cc:2229] T 686b5cf6614d45c18cc776bd79567667 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:05.688548 13188 raft_consensus.cc:2258] T 686b5cf6614d45c18cc776bd79567667 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:05.688977 9514 raft_consensus.cc:1261] T 0e4efe560ef74763b80b70665c11d7f1 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:05.689118 11655 raft_consensus.cc:2932] T 686b5cf6614d45c18cc776bd79567667 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:05.689333 13444 consensus_queue.cc:1035] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:05.689563 9908 raft_consensus.cc:1261] T 0e4efe560ef74763b80b70665c11d7f1 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:05.690131 13339 raft_consensus.cc:2932] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:05.690724 13444 consensus_queue.cc:1035] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:05.691856 9776 raft_consensus.cc:2932] T bc896a65284c48cd864bb6b57dbe5ca0 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:05.692658 13152 raft_consensus.cc:2932] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } } }
I20240502 07:59:05.693243 9644 raft_consensus.cc:2932] T bc896a65284c48cd864bb6b57dbe5ca0 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:05.693564 9908 raft_consensus.cc:2932] T 0e4efe560ef74763b80b70665c11d7f1 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } } }
I20240502 07:59:05.695992 9370 catalog_manager.cc:5484] T 0e4efe560ef74763b80b70665c11d7f1 P 6cc5b9962ed24441bcba0c07b9f5438e reported cstate change: config changed from index -1 to 4, NON_VOTER 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) added. New cstate: current_term: 1 leader_uuid: "1472c64688914149a318c7b8850a3df3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } } }
I20240502 07:59:05.696815 13188 ts_tablet_manager.cc:1875] T 686b5cf6614d45c18cc776bd79567667 P 8337af1943564c0dad18b45a597b805a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20240502 07:59:05.698343 9712 consensus_peers.cc:485] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 -> Peer 98bef2e94ade4763a66c1ad68259744a (127.3.139.71:46577): Couldn't send request to peer 98bef2e94ade4763a66c1ad68259744a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e4efe560ef74763b80b70665c11d7f1. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:05.698647 13188 ts_tablet_manager.cc:1888] T 686b5cf6614d45c18cc776bd79567667 P 8337af1943564c0dad18b45a597b805a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 07:59:05.698700 13188 log.cc:1196] T 686b5cf6614d45c18cc776bd79567667 P 8337af1943564c0dad18b45a597b805a: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal/wals/686b5cf6614d45c18cc776bd79567667
I20240502 07:59:05.699203 9358 catalog_manager.cc:4830] TS 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667): tablet 686b5cf6614d45c18cc776bd79567667 (table rebalance_test_table_0 [id=efdfc231272e43a5afefacf0ec75ec57]) successfully deleted
I20240502 07:59:05.699365 9514 raft_consensus.cc:2932] T 0e4efe560ef74763b80b70665c11d7f1 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } } }
I20240502 07:59:05.694206 9369 catalog_manager.cc:5484] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e reported cstate change: config changed from index 4 to 5, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:05.700011 11654 raft_consensus.cc:2932] T bc896a65284c48cd864bb6b57dbe5ca0 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:05.704248 9514 consensus_queue.cc:237] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } }
I20240502 07:59:05.705232 9644 raft_consensus.cc:1261] T bc896a65284c48cd864bb6b57dbe5ca0 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:05.705518 11654 raft_consensus.cc:1261] T bc896a65284c48cd864bb6b57dbe5ca0 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:05.705551 13338 consensus_queue.cc:1035] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:05.705875 13338 consensus_queue.cc:1035] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:05.707019 13338 raft_consensus.cc:2932] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:05.707749 9646 raft_consensus.cc:2932] T bc896a65284c48cd864bb6b57dbe5ca0 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:05.707783 11654 raft_consensus.cc:2932] T bc896a65284c48cd864bb6b57dbe5ca0 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:05.708698 9360 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet bc896a65284c48cd864bb6b57dbe5ca0 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:05.709250 9370 catalog_manager.cc:5484] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e reported cstate change: config changed from index 5 to 6, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:05.710242 9646 consensus_queue.cc:237] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } }
I20240502 07:59:05.711417 9757 tablet_service.cc:1514] Processing DeleteTablet for tablet bc896a65284c48cd864bb6b57dbe5ca0 with delete_type TABLET_DATA_TOMBSTONED (TS 1472c64688914149a318c7b8850a3df3 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:57656
I20240502 07:59:05.711668 9908 raft_consensus.cc:1261] T f7335ac0209c4e28b6751bb01d34d665 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:05.711889 13404 tablet_replica.cc:330] T bc896a65284c48cd864bb6b57dbe5ca0 P 1472c64688914149a318c7b8850a3df3: stopping tablet replica
I20240502 07:59:05.711879 9514 raft_consensus.cc:1261] T f7335ac0209c4e28b6751bb01d34d665 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:05.712014 13404 raft_consensus.cc:2229] T bc896a65284c48cd864bb6b57dbe5ca0 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:05.712147 13404 raft_consensus.cc:2258] T bc896a65284c48cd864bb6b57dbe5ca0 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:05.712162 13420 consensus_queue.cc:1035] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 07:59:05.712654 9582 consensus_peers.cc:485] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a -> Peer 51e82953ed2247fa9ff24638eab3508a (127.3.139.69:45345): Couldn't send request to peer 51e82953ed2247fa9ff24638eab3508a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f7335ac0209c4e28b6751bb01d34d665. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:05.712659 12641 consensus_queue.cc:1035] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:05.712936 13404 ts_tablet_manager.cc:1875] T bc896a65284c48cd864bb6b57dbe5ca0 P 1472c64688914149a318c7b8850a3df3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:05.714113 13420 raft_consensus.cc:2932] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } } }
I20240502 07:59:05.714531 9514 raft_consensus.cc:2932] T f7335ac0209c4e28b6751bb01d34d665 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } } }
I20240502 07:59:05.714785 13404 ts_tablet_manager.cc:1888] T bc896a65284c48cd864bb6b57dbe5ca0 P 1472c64688914149a318c7b8850a3df3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 07:59:05.714838 13404 log.cc:1196] T bc896a65284c48cd864bb6b57dbe5ca0 P 1472c64688914149a318c7b8850a3df3: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal/wals/bc896a65284c48cd864bb6b57dbe5ca0
I20240502 07:59:05.715267 9360 catalog_manager.cc:4830] TS 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191): tablet bc896a65284c48cd864bb6b57dbe5ca0 (table rebalance_test_table_8 [id=6aecdf32373e4c9db62c5a4844dc38ca]) successfully deleted
I20240502 07:59:05.716390 13181 raft_consensus.cc:1050] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e: attempting to promote NON_VOTER 51e82953ed2247fa9ff24638eab3508a to VOTER
I20240502 07:59:05.716732 13181 consensus_queue.cc:237] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } }
I20240502 07:59:05.717303 9367 catalog_manager.cc:5484] T f7335ac0209c4e28b6751bb01d34d665 P a201db721bda4dc3a4422dc8fa9bf72e reported cstate change: config changed from index -1 to 4, NON_VOTER 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) added. New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } } }
I20240502 07:59:05.717329 9908 raft_consensus.cc:2932] T f7335ac0209c4e28b6751bb01d34d665 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } } }
I20240502 07:59:05.718046 9776 raft_consensus.cc:1261] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.719193 13184 consensus_queue.cc:1035] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.719303 11259 raft_consensus.cc:1261] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 51e82953ed2247fa9ff24638eab3508a [term 1 LEARNER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.720063 9514 raft_consensus.cc:1261] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.720311 13184 consensus_queue.cc:1035] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.720438 13182 consensus_queue.cc:1035] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.722735 13420 raft_consensus.cc:1050] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a: attempting to promote NON_VOTER 98bef2e94ade4763a66c1ad68259744a to VOTER
I20240502 07:59:05.722841 13182 raft_consensus.cc:2932] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:05.723017 13420 consensus_queue.cc:237] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } }
I20240502 07:59:05.724292 9777 raft_consensus.cc:1261] T ecd0995c31fe4e708712c62ca60511c9 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.724527 11259 raft_consensus.cc:2932] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:05.724947 9908 raft_consensus.cc:1261] T ecd0995c31fe4e708712c62ca60511c9 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.725106 9365 catalog_manager.cc:5484] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e reported cstate change: config changed from index 4 to 5, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:05.725677 11522 raft_consensus.cc:1261] T ecd0995c31fe4e708712c62ca60511c9 P 98bef2e94ade4763a66c1ad68259744a [term 1 LEARNER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.725836 9514 raft_consensus.cc:2932] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:05.727067 13415 consensus_queue.cc:1035] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.727387 13415 consensus_queue.cc:1035] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.730285 12641 consensus_queue.cc:1035] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.731487 9776 raft_consensus.cc:2932] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:05.732057 13415 raft_consensus.cc:2932] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:05.732685 9777 raft_consensus.cc:2932] T ecd0995c31fe4e708712c62ca60511c9 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:05.733799 9906 raft_consensus.cc:2932] T ecd0995c31fe4e708712c62ca60511c9 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:05.735226 9908 consensus_queue.cc:237] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } }
I20240502 07:59:05.735312 9370 catalog_manager.cc:5484] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a reported cstate change: config changed from index 4 to 5, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:05.735502 11522 raft_consensus.cc:2932] T ecd0995c31fe4e708712c62ca60511c9 P 98bef2e94ade4763a66c1ad68259744a [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:05.736567 11259 raft_consensus.cc:1261] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:05.736946 13184 consensus_queue.cc:1035] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:05.737028 9777 raft_consensus.cc:1261] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:05.739495 13184 consensus_queue.cc:1035] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:05.740335 13184 raft_consensus.cc:2932] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:05.740867 9646 consensus_queue.cc:237] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } }
I20240502 07:59:05.742094 9776 raft_consensus.cc:1261] T ecd0995c31fe4e708712c62ca60511c9 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:05.742178 11522 raft_consensus.cc:1261] T ecd0995c31fe4e708712c62ca60511c9 P 98bef2e94ade4763a66c1ad68259744a [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:05.742228 11259 raft_consensus.cc:2932] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:05.742444 12641 consensus_queue.cc:1035] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:05.743085 12639 raft_consensus.cc:1050] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e: attempting to promote NON_VOTER 8eb9945f989d4be6b51e1531a1365350 to VOTER
I20240502 07:59:05.743403 12639 consensus_queue.cc:237] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } }
I20240502 07:59:05.744560 12641 consensus_queue.cc:1035] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:05.744684 9514 raft_consensus.cc:1261] T af07c527fff94115a743df1c73c3f4b5 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.744913 13328 raft_consensus.cc:2932] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:05.745146 9358 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 7da9d3b7fe8c40f6a68bb1b7a926d994 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:05.745307 13181 consensus_queue.cc:1035] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.745278 9776 raft_consensus.cc:2932] T ecd0995c31fe4e708712c62ca60511c9 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:05.745474 9775 raft_consensus.cc:1261] T af07c527fff94115a743df1c73c3f4b5 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.745862 9777 raft_consensus.cc:2932] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:05.746835 9358 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet ecd0995c31fe4e708712c62ca60511c9 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:05.747412 9367 catalog_manager.cc:5484] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a reported cstate change: config changed from index 5 to 6, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:05.748263 13384 raft_consensus.cc:1050] T 140868fcb19a4e7c837ce0c2c1c7041e P a201db721bda4dc3a4422dc8fa9bf72e: attempting to promote NON_VOTER 98bef2e94ade4763a66c1ad68259744a to VOTER
I20240502 07:59:05.748329 11389 raft_consensus.cc:1261] T af07c527fff94115a743df1c73c3f4b5 P 8eb9945f989d4be6b51e1531a1365350 [term 1 LEARNER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.748517 13182 consensus_queue.cc:1035] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.748598 13384 consensus_queue.cc:237] T 140868fcb19a4e7c837ce0c2c1c7041e P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } }
I20240502 07:59:05.748884 13182 consensus_queue.cc:1035] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.750169 9646 raft_consensus.cc:1261] T 140868fcb19a4e7c837ce0c2c1c7041e P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.750633 11521 raft_consensus.cc:1261] T 140868fcb19a4e7c837ce0c2c1c7041e P 98bef2e94ade4763a66c1ad68259744a [term 1 LEARNER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.750664 9365 catalog_manager.cc:5484] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 1472c64688914149a318c7b8850a3df3 reported cstate change: config changed from index 5 to 6, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New cstate: current_term: 1 leader_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:05.751089 13383 consensus_queue.cc:1035] T 140868fcb19a4e7c837ce0c2c1c7041e P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.751235 9886 tablet_service.cc:1514] Processing DeleteTablet for tablet ecd0995c31fe4e708712c62ca60511c9 with delete_type TABLET_DATA_TOMBSTONED (TS 6cc5b9962ed24441bcba0c07b9f5438e not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:41940
I20240502 07:59:05.751250 11572 consensus_queue.cc:1035] T 140868fcb19a4e7c837ce0c2c1c7041e P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.751391 13349 tablet_replica.cc:330] T ecd0995c31fe4e708712c62ca60511c9 P 6cc5b9962ed24441bcba0c07b9f5438e: stopping tablet replica
I20240502 07:59:05.751507 13349 raft_consensus.cc:2229] T ecd0995c31fe4e708712c62ca60511c9 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:05.751641 13349 raft_consensus.cc:2258] T ecd0995c31fe4e708712c62ca60511c9 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:05.753186 9776 raft_consensus.cc:1261] T 140868fcb19a4e7c837ce0c2c1c7041e P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.753513 13383 consensus_queue.cc:1035] T 140868fcb19a4e7c837ce0c2c1c7041e P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.754024 9493 tablet_service.cc:1514] Processing DeleteTablet for tablet 7da9d3b7fe8c40f6a68bb1b7a926d994 with delete_type TABLET_DATA_TOMBSTONED (TS a201db721bda4dc3a4422dc8fa9bf72e not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:41730
I20240502 07:59:05.754194 13175 tablet_replica.cc:330] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P a201db721bda4dc3a4422dc8fa9bf72e: stopping tablet replica
I20240502 07:59:05.754303 13175 raft_consensus.cc:2229] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:05.754432 13175 raft_consensus.cc:2258] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:05.755455 9646 consensus_queue.cc:237] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } }
I20240502 07:59:05.756553 9514 raft_consensus.cc:1261] T 12e3bfe638d249c0934feda41b8a8876 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:05.756616 9908 raft_consensus.cc:1261] T 12e3bfe638d249c0934feda41b8a8876 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:05.756886 13175 ts_tablet_manager.cc:1875] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P a201db721bda4dc3a4422dc8fa9bf72e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:05.756947 13420 consensus_queue.cc:1035] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:05.757066 12084 consensus_queue.cc:1035] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 07:59:05.757210 9580 consensus_peers.cc:485] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a -> Peer 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70:45329): Couldn't send request to peer 8eb9945f989d4be6b51e1531a1365350. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 12e3bfe638d249c0934feda41b8a8876. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:05.758416 13383 raft_consensus.cc:2932] T 140868fcb19a4e7c837ce0c2c1c7041e P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:05.758870 13175 ts_tablet_manager.cc:1888] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P a201db721bda4dc3a4422dc8fa9bf72e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 07:59:05.758925 13175 log.cc:1196] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P a201db721bda4dc3a4422dc8fa9bf72e: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal/wals/7da9d3b7fe8c40f6a68bb1b7a926d994
I20240502 07:59:05.758793 13420 raft_consensus.cc:2932] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } } }
I20240502 07:59:05.759656 9908 raft_consensus.cc:2932] T 12e3bfe638d249c0934feda41b8a8876 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } } }
I20240502 07:59:05.760272 9644 raft_consensus.cc:2932] T 140868fcb19a4e7c837ce0c2c1c7041e P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:05.760671 13488 raft_consensus.cc:2932] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:05.761983 13349 ts_tablet_manager.cc:1875] T ecd0995c31fe4e708712c62ca60511c9 P 6cc5b9962ed24441bcba0c07b9f5438e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:05.762748 11521 raft_consensus.cc:2932] T 140868fcb19a4e7c837ce0c2c1c7041e P 98bef2e94ade4763a66c1ad68259744a [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:05.763201 9776 raft_consensus.cc:2932] T 140868fcb19a4e7c837ce0c2c1c7041e P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:05.763371 11520 raft_consensus.cc:2932] T ecd0995c31fe4e708712c62ca60511c9 P 98bef2e94ade4763a66c1ad68259744a [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:05.763935 13349 ts_tablet_manager.cc:1888] T ecd0995c31fe4e708712c62ca60511c9 P 6cc5b9962ed24441bcba0c07b9f5438e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 07:59:05.763994 13349 log.cc:1196] T ecd0995c31fe4e708712c62ca60511c9 P 6cc5b9962ed24441bcba0c07b9f5438e: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal/wals/ecd0995c31fe4e708712c62ca60511c9
I20240502 07:59:05.764591 9358 catalog_manager.cc:4830] TS 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379): tablet ecd0995c31fe4e708712c62ca60511c9 (table rebalance_test_table_5 [id=51b1431ec18d4bb8b4d5ae1909ae1322]) successfully deleted
I20240502 07:59:05.764665 9514 raft_consensus.cc:2932] T 12e3bfe638d249c0934feda41b8a8876 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } } }
I20240502 07:59:05.766781 9360 catalog_manager.cc:4830] TS a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551): tablet 7da9d3b7fe8c40f6a68bb1b7a926d994 (table rebalance_test_table_4 [id=a6211a0188464785886f1990ca644211]) successfully deleted
I20240502 07:59:05.767215 9514 raft_consensus.cc:2932] T af07c527fff94115a743df1c73c3f4b5 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:05.768175 11389 raft_consensus.cc:2932] T af07c527fff94115a743df1c73c3f4b5 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:05.769279 9370 catalog_manager.cc:5484] T 140868fcb19a4e7c837ce0c2c1c7041e P 1472c64688914149a318c7b8850a3df3 reported cstate change: config changed from index 4 to 5, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:05.770041 13328 raft_consensus.cc:1050] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a: attempting to promote NON_VOTER a95f1d3af33f4c879fd290ca917d6329 to VOTER
I20240502 07:59:05.770363 13328 consensus_queue.cc:237] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } }
I20240502 07:59:05.770659 9365 catalog_manager.cc:5484] T af07c527fff94115a743df1c73c3f4b5 P 8eb9945f989d4be6b51e1531a1365350 reported cstate change: config changed from index 4 to 5, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:05.772394 9776 raft_consensus.cc:2932] T af07c527fff94115a743df1c73c3f4b5 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:05.772760 11654 raft_consensus.cc:1261] T d562a250cee247a48e573b25ecf00429 P a95f1d3af33f4c879fd290ca917d6329 [term 1 LEARNER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.772796 9908 raft_consensus.cc:1261] T d562a250cee247a48e573b25ecf00429 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.772809 9777 raft_consensus.cc:1261] T d562a250cee247a48e573b25ecf00429 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.773605 12084 consensus_queue.cc:1035] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.774466 13420 consensus_queue.cc:1035] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.775116 13299 consensus_queue.cc:1035] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.777925 9367 catalog_manager.cc:5484] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a reported cstate change: config changed from index -1 to 4, NON_VOTER 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) added. New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:05.780262 12641 raft_consensus.cc:2932] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:05.781488 11654 raft_consensus.cc:2932] T d562a250cee247a48e573b25ecf00429 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:05.782558 9514 consensus_queue.cc:237] T 140868fcb19a4e7c837ce0c2c1c7041e P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } }
I20240502 07:59:05.782658 9776 raft_consensus.cc:2932] T d562a250cee247a48e573b25ecf00429 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:05.783422 9906 consensus_queue.cc:237] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } }
I20240502 07:59:05.783775 9370 catalog_manager.cc:5484] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a reported cstate change: config changed from index 4 to 5, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:05.784581 11389 raft_consensus.cc:1261] T af07c527fff94115a743df1c73c3f4b5 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:05.782994 9908 raft_consensus.cc:2932] T d562a250cee247a48e573b25ecf00429 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:05.784920 13486 consensus_queue.cc:1035] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:05.786427 9644 consensus_queue.cc:237] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } }
I20240502 07:59:05.787420 11654 raft_consensus.cc:1261] T d562a250cee247a48e573b25ecf00429 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:05.787779 13299 consensus_queue.cc:1035] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:05.787917 13487 raft_consensus.cc:2932] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:05.788115 9908 raft_consensus.cc:1261] T d562a250cee247a48e573b25ecf00429 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:05.788481 13299 consensus_queue.cc:1035] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:05.788579 11520 raft_consensus.cc:1261] T 140868fcb19a4e7c837ce0c2c1c7041e P 98bef2e94ade4763a66c1ad68259744a [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:05.788563 11389 raft_consensus.cc:2932] T af07c527fff94115a743df1c73c3f4b5 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:05.788918 11572 consensus_queue.cc:1035] T 140868fcb19a4e7c837ce0c2c1c7041e P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:05.789839 9358 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet af07c527fff94115a743df1c73c3f4b5 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:05.790433 9777 raft_consensus.cc:1261] T 140868fcb19a4e7c837ce0c2c1c7041e P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:05.790627 13299 raft_consensus.cc:2932] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:05.790792 11572 consensus_queue.cc:1035] T 140868fcb19a4e7c837ce0c2c1c7041e P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:05.790741 9364 catalog_manager.cc:5484] T af07c527fff94115a743df1c73c3f4b5 P 8eb9945f989d4be6b51e1531a1365350 reported cstate change: config changed from index 5 to 6, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New cstate: current_term: 1 leader_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:05.791996 9358 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet d562a250cee247a48e573b25ecf00429 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:05.792443 13339 raft_consensus.cc:2932] T 140868fcb19a4e7c837ce0c2c1c7041e P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:05.792809 9777 raft_consensus.cc:2932] T 140868fcb19a4e7c837ce0c2c1c7041e P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:05.792891 9493 tablet_service.cc:1514] Processing DeleteTablet for tablet af07c527fff94115a743df1c73c3f4b5 with delete_type TABLET_DATA_TOMBSTONED (TS a201db721bda4dc3a4422dc8fa9bf72e not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:41730
I20240502 07:59:05.793056 13175 tablet_replica.cc:330] T af07c527fff94115a743df1c73c3f4b5 P a201db721bda4dc3a4422dc8fa9bf72e: stopping tablet replica
I20240502 07:59:05.793169 13175 raft_consensus.cc:2229] T af07c527fff94115a743df1c73c3f4b5 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:05.793294 13175 raft_consensus.cc:2258] T af07c527fff94115a743df1c73c3f4b5 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:05.793736 9367 catalog_manager.cc:5484] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a reported cstate change: config changed from index 5 to 6, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:05.793884 13175 ts_tablet_manager.cc:1875] T af07c527fff94115a743df1c73c3f4b5 P a201db721bda4dc3a4422dc8fa9bf72e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:05.794447 9908 raft_consensus.cc:2932] T d562a250cee247a48e573b25ecf00429 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:05.794692 11520 raft_consensus.cc:2932] T 140868fcb19a4e7c837ce0c2c1c7041e P 98bef2e94ade4763a66c1ad68259744a [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:05.795657 13175 ts_tablet_manager.cc:1888] T af07c527fff94115a743df1c73c3f4b5 P a201db721bda4dc3a4422dc8fa9bf72e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 07:59:05.795713 13175 log.cc:1196] T af07c527fff94115a743df1c73c3f4b5 P a201db721bda4dc3a4422dc8fa9bf72e: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal/wals/af07c527fff94115a743df1c73c3f4b5
I20240502 07:59:05.796458 9360 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 140868fcb19a4e7c837ce0c2c1c7041e with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:05.796594 9360 catalog_manager.cc:4830] TS a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551): tablet af07c527fff94115a743df1c73c3f4b5 (table rebalance_test_table_6 [id=af6c4a01744d4fcfbdf53cfc8d19500b]) successfully deleted
I20240502 07:59:05.797003 9365 catalog_manager.cc:5484] T 140868fcb19a4e7c837ce0c2c1c7041e P 98bef2e94ade4763a66c1ad68259744a reported cstate change: config changed from index 5 to 6, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:05.797787 9776 raft_consensus.cc:1261] T af07c527fff94115a743df1c73c3f4b5 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:05.798159 13488 consensus_queue.cc:1035] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:05.799418 9776 raft_consensus.cc:2932] T af07c527fff94115a743df1c73c3f4b5 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:05.800489 9624 tablet_service.cc:1514] Processing DeleteTablet for tablet 140868fcb19a4e7c837ce0c2c1c7041e with delete_type TABLET_DATA_TOMBSTONED (TS 8337af1943564c0dad18b45a597b805a not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:53862
I20240502 07:59:05.800659 13188 tablet_replica.cc:330] T 140868fcb19a4e7c837ce0c2c1c7041e P 8337af1943564c0dad18b45a597b805a: stopping tablet replica
I20240502 07:59:05.800784 13188 raft_consensus.cc:2229] T 140868fcb19a4e7c837ce0c2c1c7041e P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:05.800909 13188 raft_consensus.cc:2258] T 140868fcb19a4e7c837ce0c2c1c7041e P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:05.801486 13188 ts_tablet_manager.cc:1875] T 140868fcb19a4e7c837ce0c2c1c7041e P 8337af1943564c0dad18b45a597b805a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:05.802127 11654 raft_consensus.cc:2932] T d562a250cee247a48e573b25ecf00429 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:05.803422 13188 ts_tablet_manager.cc:1888] T 140868fcb19a4e7c837ce0c2c1c7041e P 8337af1943564c0dad18b45a597b805a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 07:59:05.803478 13188 log.cc:1196] T 140868fcb19a4e7c837ce0c2c1c7041e P 8337af1943564c0dad18b45a597b805a: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal/wals/140868fcb19a4e7c837ce0c2c1c7041e
I20240502 07:59:05.804862 9358 catalog_manager.cc:4830] TS 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667): tablet 140868fcb19a4e7c837ce0c2c1c7041e (table rebalance_test_table_4 [id=a6211a0188464785886f1990ca644211]) successfully deleted
I20240502 07:59:05.805462 9757 tablet_service.cc:1514] Processing DeleteTablet for tablet d562a250cee247a48e573b25ecf00429 with delete_type TABLET_DATA_TOMBSTONED (TS 1472c64688914149a318c7b8850a3df3 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:57656
I20240502 07:59:05.805647 13404 tablet_replica.cc:330] T d562a250cee247a48e573b25ecf00429 P 1472c64688914149a318c7b8850a3df3: stopping tablet replica
I20240502 07:59:05.805754 13404 raft_consensus.cc:2229] T d562a250cee247a48e573b25ecf00429 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:05.805886 13404 raft_consensus.cc:2258] T d562a250cee247a48e573b25ecf00429 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:05.806512 13404 ts_tablet_manager.cc:1875] T d562a250cee247a48e573b25ecf00429 P 1472c64688914149a318c7b8850a3df3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:05.808269 13404 ts_tablet_manager.cc:1888] T d562a250cee247a48e573b25ecf00429 P 1472c64688914149a318c7b8850a3df3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 07:59:05.808331 13404 log.cc:1196] T d562a250cee247a48e573b25ecf00429 P 1472c64688914149a318c7b8850a3df3: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal/wals/d562a250cee247a48e573b25ecf00429
I20240502 07:59:05.808815 9360 catalog_manager.cc:4830] TS 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191): tablet d562a250cee247a48e573b25ecf00429 (table rebalance_test_table_1 [id=982b41384c8f44b99bbe8eecffb1cd87]) successfully deleted
I20240502 07:59:05.818807 13439 raft_consensus.cc:1050] T d3db04c54a184eb8a1c23fec9e7145eb P 1472c64688914149a318c7b8850a3df3: attempting to promote NON_VOTER 51e82953ed2247fa9ff24638eab3508a to VOTER
I20240502 07:59:05.819226 13439 consensus_queue.cc:237] T d3db04c54a184eb8a1c23fec9e7145eb P 1472c64688914149a318c7b8850a3df3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } }
I20240502 07:59:05.820466 9644 raft_consensus.cc:1261] T d3db04c54a184eb8a1c23fec9e7145eb P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.820497 11259 raft_consensus.cc:1261] T d3db04c54a184eb8a1c23fec9e7145eb P 51e82953ed2247fa9ff24638eab3508a [term 1 LEARNER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.820771 9908 raft_consensus.cc:1261] T d3db04c54a184eb8a1c23fec9e7145eb P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.820833 13463 consensus_queue.cc:1035] T d3db04c54a184eb8a1c23fec9e7145eb P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.821017 13439 consensus_queue.cc:1035] T d3db04c54a184eb8a1c23fec9e7145eb P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.821156 13444 consensus_queue.cc:1035] T d3db04c54a184eb8a1c23fec9e7145eb P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.822914 13152 raft_consensus.cc:2932] T d3db04c54a184eb8a1c23fec9e7145eb P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:05.823354 9908 raft_consensus.cc:2932] T d3db04c54a184eb8a1c23fec9e7145eb P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:05.824563 9644 raft_consensus.cc:2932] T d3db04c54a184eb8a1c23fec9e7145eb P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:05.824894 11259 raft_consensus.cc:2932] T d3db04c54a184eb8a1c23fec9e7145eb P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:05.825163 9367 catalog_manager.cc:5484] T d3db04c54a184eb8a1c23fec9e7145eb P 1472c64688914149a318c7b8850a3df3 reported cstate change: config changed from index 4 to 5, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1472c64688914149a318c7b8850a3df3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:05.829253 9776 consensus_queue.cc:237] T d3db04c54a184eb8a1c23fec9e7145eb P 1472c64688914149a318c7b8850a3df3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } }
I20240502 07:59:05.830159 11259 raft_consensus.cc:1261] T d3db04c54a184eb8a1c23fec9e7145eb P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:05.830494 9908 raft_consensus.cc:1261] T d3db04c54a184eb8a1c23fec9e7145eb P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:05.830543 13152 consensus_queue.cc:1035] T d3db04c54a184eb8a1c23fec9e7145eb P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:05.831164 13152 consensus_queue.cc:1035] T d3db04c54a184eb8a1c23fec9e7145eb P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:05.832525 13439 raft_consensus.cc:2932] T d3db04c54a184eb8a1c23fec9e7145eb P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:05.832947 11259 raft_consensus.cc:2932] T d3db04c54a184eb8a1c23fec9e7145eb P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:05.833014 9908 raft_consensus.cc:2932] T d3db04c54a184eb8a1c23fec9e7145eb P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:05.833691 9360 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet d3db04c54a184eb8a1c23fec9e7145eb with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:05.835165 9370 catalog_manager.cc:5484] T d3db04c54a184eb8a1c23fec9e7145eb P 51e82953ed2247fa9ff24638eab3508a reported cstate change: config changed from index 5 to 6, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New cstate: current_term: 1 leader_uuid: "1472c64688914149a318c7b8850a3df3" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:05.837359 9624 tablet_service.cc:1514] Processing DeleteTablet for tablet d3db04c54a184eb8a1c23fec9e7145eb with delete_type TABLET_DATA_TOMBSTONED (TS 8337af1943564c0dad18b45a597b805a not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:53862
I20240502 07:59:05.837570 13188 tablet_replica.cc:330] T d3db04c54a184eb8a1c23fec9e7145eb P 8337af1943564c0dad18b45a597b805a: stopping tablet replica
I20240502 07:59:05.837703 13188 raft_consensus.cc:2229] T d3db04c54a184eb8a1c23fec9e7145eb P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:05.837864 13188 raft_consensus.cc:2258] T d3db04c54a184eb8a1c23fec9e7145eb P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:05.838465 13188 ts_tablet_manager.cc:1875] T d3db04c54a184eb8a1c23fec9e7145eb P 8337af1943564c0dad18b45a597b805a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:05.840059 13188 ts_tablet_manager.cc:1888] T d3db04c54a184eb8a1c23fec9e7145eb P 8337af1943564c0dad18b45a597b805a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 07:59:05.840118 13188 log.cc:1196] T d3db04c54a184eb8a1c23fec9e7145eb P 8337af1943564c0dad18b45a597b805a: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal/wals/d3db04c54a184eb8a1c23fec9e7145eb
I20240502 07:59:05.840524 9358 catalog_manager.cc:4830] TS 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667): tablet d3db04c54a184eb8a1c23fec9e7145eb (table rebalance_test_table_5 [id=51b1431ec18d4bb8b4d5ae1909ae1322]) successfully deleted
I20240502 07:59:05.897805 13587 ts_tablet_manager.cc:902] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8eb9945f989d4be6b51e1531a1365350: Initiating tablet copy from peer 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667)
I20240502 07:59:05.898298 13587 tablet_copy_client.cc:308] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.66:46667
I20240502 07:59:05.898713 9666 tablet_copy_service.cc:140] P 8337af1943564c0dad18b45a597b805a: Received BeginTabletCopySession request for tablet 427c037fa09c4b94b7bc6c71bdb61c2f from peer 8eb9945f989d4be6b51e1531a1365350 ({username='slave'} at 127.3.139.70:48793)
I20240502 07:59:05.898821 9666 tablet_copy_service.cc:161] P 8337af1943564c0dad18b45a597b805a: Beginning new tablet copy session on tablet 427c037fa09c4b94b7bc6c71bdb61c2f from peer 8eb9945f989d4be6b51e1531a1365350 at {username='slave'} at 127.3.139.70:48793: session id = 8eb9945f989d4be6b51e1531a1365350-427c037fa09c4b94b7bc6c71bdb61c2f
I20240502 07:59:05.899680 9666 tablet_copy_source_session.cc:205] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:05.900477 13587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 427c037fa09c4b94b7bc6c71bdb61c2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:05.902794 13587 tablet_copy_client.cc:783] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:05.902923 13587 tablet_copy_client.cc:647] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:05.903920 13591 tablet_copy_client.cc:1002] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Injecting latency into file download: 1500
I20240502 07:59:05.908668 9776 consensus_queue.cc:237] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } }
I20240502 07:59:05.909911 9514 raft_consensus.cc:1261] T d37b8a7f016647a7a03baedc37ed5419 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:05.910256 13439 consensus_queue.cc:1035] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:05.910841 9644 raft_consensus.cc:1261] T d37b8a7f016647a7a03baedc37ed5419 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:05.911229 13439 consensus_queue.cc:1035] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 07:59:05.911418 9712 consensus_peers.cc:485] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 -> Peer a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72:36271): Couldn't send request to peer a95f1d3af33f4c879fd290ca917d6329. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d37b8a7f016647a7a03baedc37ed5419. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:05.912725 13444 raft_consensus.cc:2932] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } } }
I20240502 07:59:05.913085 9644 raft_consensus.cc:2932] T d37b8a7f016647a7a03baedc37ed5419 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } } }
I20240502 07:59:05.915287 9369 catalog_manager.cc:5484] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 reported cstate change: config changed from index -1 to 4, NON_VOTER a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) added. New cstate: current_term: 1 leader_uuid: "1472c64688914149a318c7b8850a3df3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:05.922097 9515 consensus_queue.cc:237] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } }
I20240502 07:59:05.923460 9776 raft_consensus.cc:1261] T 265eca6fff644d83857ed48a04864a58 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:05.923583 9908 raft_consensus.cc:1261] T 265eca6fff644d83857ed48a04864a58 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 07:59:05.923666 9450 consensus_peers.cc:485] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e -> Peer a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72:36271): Couldn't send request to peer a95f1d3af33f4c879fd290ca917d6329. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 265eca6fff644d83857ed48a04864a58. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:05.923907 13391 consensus_queue.cc:1035] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:05.924060 13337 consensus_queue.cc:1035] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:05.925577 13337 raft_consensus.cc:2932] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } } }
I20240502 07:59:05.926026 9776 raft_consensus.cc:2932] T 265eca6fff644d83857ed48a04864a58 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } } }
I20240502 07:59:05.927807 9367 catalog_manager.cc:5484] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e reported cstate change: config changed from index -1 to 4, NON_VOTER a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) added. New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:05.927820 9908 raft_consensus.cc:2932] T 265eca6fff644d83857ed48a04864a58 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } } }
I20240502 07:59:05.930748 9514 raft_consensus.cc:2932] T d37b8a7f016647a7a03baedc37ed5419 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } } }
I20240502 07:59:05.939539 9776 consensus_queue.cc:237] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } }
I20240502 07:59:05.940619 9908 raft_consensus.cc:1261] T 70d4db3c3291490db58678522d356688 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:05.940708 9644 raft_consensus.cc:1261] T 70d4db3c3291490db58678522d356688 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 07:59:05.940968 9711 consensus_peers.cc:485] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 -> Peer 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70:45329): Couldn't send request to peer 8eb9945f989d4be6b51e1531a1365350. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 70d4db3c3291490db58678522d356688. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:05.941112 13463 consensus_queue.cc:1035] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:05.941264 13439 consensus_queue.cc:1035] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:05.942879 13439 raft_consensus.cc:2932] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } } }
I20240502 07:59:05.943312 9644 raft_consensus.cc:2932] T 70d4db3c3291490db58678522d356688 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } } }
I20240502 07:59:05.945149 9369 catalog_manager.cc:5484] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 reported cstate change: config changed from index -1 to 4, NON_VOTER 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) added. New cstate: current_term: 1 leader_uuid: "1472c64688914149a318c7b8850a3df3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:05.945525 9908 raft_consensus.cc:2932] T 70d4db3c3291490db58678522d356688 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } } }
I20240502 07:59:05.953439 9514 consensus_queue.cc:237] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } }
I20240502 07:59:05.954643 9644 raft_consensus.cc:1261] T ee2e79e677f643409e3c8dd49b9517cf P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:05.954660 9908 raft_consensus.cc:1261] T ee2e79e677f643409e3c8dd49b9517cf P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 07:59:05.954914 9450 consensus_peers.cc:485] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e -> Peer a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72:36271): Couldn't send request to peer a95f1d3af33f4c879fd290ca917d6329. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ee2e79e677f643409e3c8dd49b9517cf. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:05.955092 13384 consensus_queue.cc:1035] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:05.955257 11572 consensus_queue.cc:1035] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:05.956969 13391 raft_consensus.cc:2932] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } } }
I20240502 07:59:05.957242 9644 raft_consensus.cc:2932] T ee2e79e677f643409e3c8dd49b9517cf P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } } }
I20240502 07:59:05.957298 9908 raft_consensus.cc:2932] T ee2e79e677f643409e3c8dd49b9517cf P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } } }
I20240502 07:59:05.959470 9367 catalog_manager.cc:5484] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e reported cstate change: config changed from index -1 to 4, NON_VOTER a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) added. New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:05.966504 9514 consensus_queue.cc:237] T 8cf1917de461431d98c5155db3eb2312 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } }
I20240502 07:59:05.967726 9644 raft_consensus.cc:1261] T 8cf1917de461431d98c5155db3eb2312 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:05.967806 9776 raft_consensus.cc:1261] T 8cf1917de461431d98c5155db3eb2312 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 07:59:05.967932 9450 consensus_peers.cc:485] T 8cf1917de461431d98c5155db3eb2312 P a201db721bda4dc3a4422dc8fa9bf72e -> Peer a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72:36271): Couldn't send request to peer a95f1d3af33f4c879fd290ca917d6329. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8cf1917de461431d98c5155db3eb2312. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:05.968053 11572 consensus_queue.cc:1035] T 8cf1917de461431d98c5155db3eb2312 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:05.968204 13391 consensus_queue.cc:1035] T 8cf1917de461431d98c5155db3eb2312 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:05.969744 13384 raft_consensus.cc:2932] T 8cf1917de461431d98c5155db3eb2312 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } } }
I20240502 07:59:05.970288 9644 raft_consensus.cc:2932] T 8cf1917de461431d98c5155db3eb2312 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } } }
I20240502 07:59:05.972152 9370 catalog_manager.cc:5484] T 8cf1917de461431d98c5155db3eb2312 P a201db721bda4dc3a4422dc8fa9bf72e reported cstate change: config changed from index -1 to 4, NON_VOTER a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) added. New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:05.972235 9776 raft_consensus.cc:2932] T 8cf1917de461431d98c5155db3eb2312 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } } }
I20240502 07:59:06.073930 13391 raft_consensus.cc:1050] T e2ba8220f81a4d6cb3611ca53db91fe1 P a201db721bda4dc3a4422dc8fa9bf72e: attempting to promote NON_VOTER 98bef2e94ade4763a66c1ad68259744a to VOTER
I20240502 07:59:06.074318 13391 consensus_queue.cc:237] T e2ba8220f81a4d6cb3611ca53db91fe1 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } }
I20240502 07:59:06.075634 9776 raft_consensus.cc:1261] T e2ba8220f81a4d6cb3611ca53db91fe1 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:06.075634 9644 raft_consensus.cc:1261] T e2ba8220f81a4d6cb3611ca53db91fe1 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:06.075872 11520 raft_consensus.cc:1261] T e2ba8220f81a4d6cb3611ca53db91fe1 P 98bef2e94ade4763a66c1ad68259744a [term 1 LEARNER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:06.076166 13338 consensus_queue.cc:1035] T e2ba8220f81a4d6cb3611ca53db91fe1 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:06.076395 13391 consensus_queue.cc:1035] T e2ba8220f81a4d6cb3611ca53db91fe1 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:06.076566 13337 consensus_queue.cc:1035] T e2ba8220f81a4d6cb3611ca53db91fe1 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:06.078490 13391 raft_consensus.cc:2932] T e2ba8220f81a4d6cb3611ca53db91fe1 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:06.078881 9776 raft_consensus.cc:2932] T e2ba8220f81a4d6cb3611ca53db91fe1 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:06.080232 11520 raft_consensus.cc:2932] T e2ba8220f81a4d6cb3611ca53db91fe1 P 98bef2e94ade4763a66c1ad68259744a [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:06.082468 9367 catalog_manager.cc:5484] T e2ba8220f81a4d6cb3611ca53db91fe1 P 1472c64688914149a318c7b8850a3df3 reported cstate change: config changed from index 4 to 5, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:06.083570 9644 raft_consensus.cc:2932] T e2ba8220f81a4d6cb3611ca53db91fe1 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:06.087304 9514 consensus_queue.cc:237] T e2ba8220f81a4d6cb3611ca53db91fe1 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } }
I20240502 07:59:06.088552 9776 raft_consensus.cc:1261] T e2ba8220f81a4d6cb3611ca53db91fe1 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:06.089004 13337 consensus_queue.cc:1035] T e2ba8220f81a4d6cb3611ca53db91fe1 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:06.089645 11520 raft_consensus.cc:1261] T e2ba8220f81a4d6cb3611ca53db91fe1 P 98bef2e94ade4763a66c1ad68259744a [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:06.090188 11572 consensus_queue.cc:1035] T e2ba8220f81a4d6cb3611ca53db91fe1 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:06.090518 11572 raft_consensus.cc:2932] T e2ba8220f81a4d6cb3611ca53db91fe1 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:06.091110 9776 raft_consensus.cc:2932] T e2ba8220f81a4d6cb3611ca53db91fe1 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:06.091936 9360 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet e2ba8220f81a4d6cb3611ca53db91fe1 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:06.093084 9370 catalog_manager.cc:5484] T e2ba8220f81a4d6cb3611ca53db91fe1 P a201db721bda4dc3a4422dc8fa9bf72e reported cstate change: config changed from index 5 to 6, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:06.093809 11520 raft_consensus.cc:2932] T e2ba8220f81a4d6cb3611ca53db91fe1 P 98bef2e94ade4763a66c1ad68259744a [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:06.095185 9624 tablet_service.cc:1514] Processing DeleteTablet for tablet e2ba8220f81a4d6cb3611ca53db91fe1 with delete_type TABLET_DATA_TOMBSTONED (TS 8337af1943564c0dad18b45a597b805a not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:53862
I20240502 07:59:06.095376 13188 tablet_replica.cc:330] T e2ba8220f81a4d6cb3611ca53db91fe1 P 8337af1943564c0dad18b45a597b805a: stopping tablet replica
I20240502 07:59:06.095512 13188 raft_consensus.cc:2229] T e2ba8220f81a4d6cb3611ca53db91fe1 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:06.095670 13188 raft_consensus.cc:2258] T e2ba8220f81a4d6cb3611ca53db91fe1 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:06.096609 13188 ts_tablet_manager.cc:1875] T e2ba8220f81a4d6cb3611ca53db91fe1 P 8337af1943564c0dad18b45a597b805a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:06.098503 13188 ts_tablet_manager.cc:1888] T e2ba8220f81a4d6cb3611ca53db91fe1 P 8337af1943564c0dad18b45a597b805a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 07:59:06.098569 13188 log.cc:1196] T e2ba8220f81a4d6cb3611ca53db91fe1 P 8337af1943564c0dad18b45a597b805a: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal/wals/e2ba8220f81a4d6cb3611ca53db91fe1
I20240502 07:59:06.098981 9358 catalog_manager.cc:4830] TS 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667): tablet e2ba8220f81a4d6cb3611ca53db91fe1 (table rebalance_test_table_1 [id=982b41384c8f44b99bbe8eecffb1cd87]) successfully deleted
I20240502 07:59:06.122258 13691 ts_tablet_manager.cc:902] T 0e4efe560ef74763b80b70665c11d7f1 P 98bef2e94ade4763a66c1ad68259744a: Initiating tablet copy from peer 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191)
I20240502 07:59:06.122709 13691 tablet_copy_client.cc:308] T 0e4efe560ef74763b80b70665c11d7f1 P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.67:43191
I20240502 07:59:06.126010 9797 tablet_copy_service.cc:140] P 1472c64688914149a318c7b8850a3df3: Received BeginTabletCopySession request for tablet 0e4efe560ef74763b80b70665c11d7f1 from peer 98bef2e94ade4763a66c1ad68259744a ({username='slave'} at 127.3.139.71:56869)
I20240502 07:59:06.126094 9797 tablet_copy_service.cc:161] P 1472c64688914149a318c7b8850a3df3: Beginning new tablet copy session on tablet 0e4efe560ef74763b80b70665c11d7f1 from peer 98bef2e94ade4763a66c1ad68259744a at {username='slave'} at 127.3.139.71:56869: session id = 98bef2e94ade4763a66c1ad68259744a-0e4efe560ef74763b80b70665c11d7f1
I20240502 07:59:06.127394 9797 tablet_copy_source_session.cc:205] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:06.129328 13691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e4efe560ef74763b80b70665c11d7f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:06.131161 13691 tablet_copy_client.cc:783] T 0e4efe560ef74763b80b70665c11d7f1 P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:06.131319 13691 tablet_copy_client.cc:647] T 0e4efe560ef74763b80b70665c11d7f1 P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:06.132087 13695 tablet_copy_client.cc:1002] T 0e4efe560ef74763b80b70665c11d7f1 P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Injecting latency into file download: 1500
I20240502 07:59:06.213662 13697 ts_tablet_manager.cc:902] T f7335ac0209c4e28b6751bb01d34d665 P 51e82953ed2247fa9ff24638eab3508a: Initiating tablet copy from peer 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667)
I20240502 07:59:06.214102 13697 tablet_copy_client.cc:308] T f7335ac0209c4e28b6751bb01d34d665 P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.66:46667
I20240502 07:59:06.214576 9666 tablet_copy_service.cc:140] P 8337af1943564c0dad18b45a597b805a: Received BeginTabletCopySession request for tablet f7335ac0209c4e28b6751bb01d34d665 from peer 51e82953ed2247fa9ff24638eab3508a ({username='slave'} at 127.3.139.69:35563)
I20240502 07:59:06.214666 9666 tablet_copy_service.cc:161] P 8337af1943564c0dad18b45a597b805a: Beginning new tablet copy session on tablet f7335ac0209c4e28b6751bb01d34d665 from peer 51e82953ed2247fa9ff24638eab3508a at {username='slave'} at 127.3.139.69:35563: session id = 51e82953ed2247fa9ff24638eab3508a-f7335ac0209c4e28b6751bb01d34d665
I20240502 07:59:06.215359 9666 tablet_copy_source_session.cc:205] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:06.216086 13697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7335ac0209c4e28b6751bb01d34d665. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:06.218096 13697 tablet_copy_client.cc:783] T f7335ac0209c4e28b6751bb01d34d665 P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:06.218251 13697 tablet_copy_client.cc:647] T f7335ac0209c4e28b6751bb01d34d665 P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:06.219206 13701 tablet_copy_client.cc:1002] T f7335ac0209c4e28b6751bb01d34d665 P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Injecting latency into file download: 1500
I20240502 07:59:06.299882 13782 ts_tablet_manager.cc:902] T d37b8a7f016647a7a03baedc37ed5419 P a95f1d3af33f4c879fd290ca917d6329: Initiating tablet copy from peer 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191)
I20240502 07:59:06.300400 13782 tablet_copy_client.cc:308] T d37b8a7f016647a7a03baedc37ed5419 P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.67:43191
I20240502 07:59:06.300837 9797 tablet_copy_service.cc:140] P 1472c64688914149a318c7b8850a3df3: Received BeginTabletCopySession request for tablet d37b8a7f016647a7a03baedc37ed5419 from peer a95f1d3af33f4c879fd290ca917d6329 ({username='slave'} at 127.3.139.72:51819)
I20240502 07:59:06.300917 9797 tablet_copy_service.cc:161] P 1472c64688914149a318c7b8850a3df3: Beginning new tablet copy session on tablet d37b8a7f016647a7a03baedc37ed5419 from peer a95f1d3af33f4c879fd290ca917d6329 at {username='slave'} at 127.3.139.72:51819: session id = a95f1d3af33f4c879fd290ca917d6329-d37b8a7f016647a7a03baedc37ed5419
I20240502 07:59:06.301648 9797 tablet_copy_source_session.cc:205] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:06.302444 13782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d37b8a7f016647a7a03baedc37ed5419. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:06.304464 13782 tablet_copy_client.cc:783] T d37b8a7f016647a7a03baedc37ed5419 P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:06.304724 13782 tablet_copy_client.cc:647] T d37b8a7f016647a7a03baedc37ed5419 P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:06.305608 13786 tablet_copy_client.cc:1002] T d37b8a7f016647a7a03baedc37ed5419 P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Injecting latency into file download: 1500
I20240502 07:59:06.324048 13803 ts_tablet_manager.cc:902] T 70d4db3c3291490db58678522d356688 P 8eb9945f989d4be6b51e1531a1365350: Initiating tablet copy from peer 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191)
I20240502 07:59:06.324487 13803 tablet_copy_client.cc:308] T 70d4db3c3291490db58678522d356688 P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.67:43191
I20240502 07:59:06.324877 9797 tablet_copy_service.cc:140] P 1472c64688914149a318c7b8850a3df3: Received BeginTabletCopySession request for tablet 70d4db3c3291490db58678522d356688 from peer 8eb9945f989d4be6b51e1531a1365350 ({username='slave'} at 127.3.139.70:41973)
I20240502 07:59:06.324950 9797 tablet_copy_service.cc:161] P 1472c64688914149a318c7b8850a3df3: Beginning new tablet copy session on tablet 70d4db3c3291490db58678522d356688 from peer 8eb9945f989d4be6b51e1531a1365350 at {username='slave'} at 127.3.139.70:41973: session id = 8eb9945f989d4be6b51e1531a1365350-70d4db3c3291490db58678522d356688
I20240502 07:59:06.325503 9797 tablet_copy_source_session.cc:205] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:06.326133 13803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70d4db3c3291490db58678522d356688. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:06.327812 13803 tablet_copy_client.cc:783] T 70d4db3c3291490db58678522d356688 P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:06.327944 13803 tablet_copy_client.cc:647] T 70d4db3c3291490db58678522d356688 P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:06.328749 13806 tablet_copy_client.cc:1002] T 70d4db3c3291490db58678522d356688 P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Injecting latency into file download: 1500
I20240502 07:59:06.329949 13808 ts_tablet_manager.cc:902] T 12e3bfe638d249c0934feda41b8a8876 P 8eb9945f989d4be6b51e1531a1365350: Initiating tablet copy from peer 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667)
I20240502 07:59:06.330353 13808 tablet_copy_client.cc:308] T 12e3bfe638d249c0934feda41b8a8876 P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.66:46667
I20240502 07:59:06.330734 9666 tablet_copy_service.cc:140] P 8337af1943564c0dad18b45a597b805a: Received BeginTabletCopySession request for tablet 12e3bfe638d249c0934feda41b8a8876 from peer 8eb9945f989d4be6b51e1531a1365350 ({username='slave'} at 127.3.139.70:48793)
I20240502 07:59:06.330814 9666 tablet_copy_service.cc:161] P 8337af1943564c0dad18b45a597b805a: Beginning new tablet copy session on tablet 12e3bfe638d249c0934feda41b8a8876 from peer 8eb9945f989d4be6b51e1531a1365350 at {username='slave'} at 127.3.139.70:48793: session id = 8eb9945f989d4be6b51e1531a1365350-12e3bfe638d249c0934feda41b8a8876
I20240502 07:59:06.331425 9666 tablet_copy_source_session.cc:205] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:06.332028 13808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12e3bfe638d249c0934feda41b8a8876. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:06.333427 13808 tablet_copy_client.cc:783] T 12e3bfe638d249c0934feda41b8a8876 P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:06.333546 13808 tablet_copy_client.cc:647] T 12e3bfe638d249c0934feda41b8a8876 P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:06.334339 13812 tablet_copy_client.cc:1002] T 12e3bfe638d249c0934feda41b8a8876 P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Injecting latency into file download: 1500
I20240502 07:59:06.435575 13813 ts_tablet_manager.cc:902] T 265eca6fff644d83857ed48a04864a58 P a95f1d3af33f4c879fd290ca917d6329: Initiating tablet copy from peer a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551)
I20240502 07:59:06.435989 13813 tablet_copy_client.cc:308] T 265eca6fff644d83857ed48a04864a58 P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.65:39551
I20240502 07:59:06.436399 9535 tablet_copy_service.cc:140] P a201db721bda4dc3a4422dc8fa9bf72e: Received BeginTabletCopySession request for tablet 265eca6fff644d83857ed48a04864a58 from peer a95f1d3af33f4c879fd290ca917d6329 ({username='slave'} at 127.3.139.72:38421)
I20240502 07:59:06.436491 9535 tablet_copy_service.cc:161] P a201db721bda4dc3a4422dc8fa9bf72e: Beginning new tablet copy session on tablet 265eca6fff644d83857ed48a04864a58 from peer a95f1d3af33f4c879fd290ca917d6329 at {username='slave'} at 127.3.139.72:38421: session id = a95f1d3af33f4c879fd290ca917d6329-265eca6fff644d83857ed48a04864a58
I20240502 07:59:06.437156 9535 tablet_copy_source_session.cc:205] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:06.437871 13813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 265eca6fff644d83857ed48a04864a58. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:06.439410 13813 tablet_copy_client.cc:783] T 265eca6fff644d83857ed48a04864a58 P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:06.439538 13813 tablet_copy_client.cc:647] T 265eca6fff644d83857ed48a04864a58 P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:06.440428 13817 tablet_copy_client.cc:1002] T 265eca6fff644d83857ed48a04864a58 P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Injecting latency into file download: 1500
I20240502 07:59:06.567481 13864 ts_tablet_manager.cc:902] T 8cf1917de461431d98c5155db3eb2312 P a95f1d3af33f4c879fd290ca917d6329: Initiating tablet copy from peer a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551)
I20240502 07:59:06.567981 13864 tablet_copy_client.cc:308] T 8cf1917de461431d98c5155db3eb2312 P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.65:39551
I20240502 07:59:06.568743 13869 ts_tablet_manager.cc:902] T ee2e79e677f643409e3c8dd49b9517cf P a95f1d3af33f4c879fd290ca917d6329: Initiating tablet copy from peer a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551)
I20240502 07:59:06.568750 9535 tablet_copy_service.cc:140] P a201db721bda4dc3a4422dc8fa9bf72e: Received BeginTabletCopySession request for tablet 8cf1917de461431d98c5155db3eb2312 from peer a95f1d3af33f4c879fd290ca917d6329 ({username='slave'} at 127.3.139.72:38421)
I20240502 07:59:06.568861 9535 tablet_copy_service.cc:161] P a201db721bda4dc3a4422dc8fa9bf72e: Beginning new tablet copy session on tablet 8cf1917de461431d98c5155db3eb2312 from peer a95f1d3af33f4c879fd290ca917d6329 at {username='slave'} at 127.3.139.72:38421: session id = a95f1d3af33f4c879fd290ca917d6329-8cf1917de461431d98c5155db3eb2312
I20240502 07:59:06.569223 13869 tablet_copy_client.cc:308] T ee2e79e677f643409e3c8dd49b9517cf P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.65:39551
I20240502 07:59:06.569623 9534 tablet_copy_service.cc:140] P a201db721bda4dc3a4422dc8fa9bf72e: Received BeginTabletCopySession request for tablet ee2e79e677f643409e3c8dd49b9517cf from peer a95f1d3af33f4c879fd290ca917d6329 ({username='slave'} at 127.3.139.72:38421)
I20240502 07:59:06.569718 9534 tablet_copy_service.cc:161] P a201db721bda4dc3a4422dc8fa9bf72e: Beginning new tablet copy session on tablet ee2e79e677f643409e3c8dd49b9517cf from peer a95f1d3af33f4c879fd290ca917d6329 at {username='slave'} at 127.3.139.72:38421: session id = a95f1d3af33f4c879fd290ca917d6329-ee2e79e677f643409e3c8dd49b9517cf
I20240502 07:59:06.569751 9535 tablet_copy_source_session.cc:205] T 8cf1917de461431d98c5155db3eb2312 P a201db721bda4dc3a4422dc8fa9bf72e: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:06.570545 13864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cf1917de461431d98c5155db3eb2312. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:06.570561 9534 tablet_copy_source_session.cc:205] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:06.571256 13869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee2e79e677f643409e3c8dd49b9517cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:06.572540 13864 tablet_copy_client.cc:783] T 8cf1917de461431d98c5155db3eb2312 P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:06.572675 13864 tablet_copy_client.cc:647] T 8cf1917de461431d98c5155db3eb2312 P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:06.572902 13869 tablet_copy_client.cc:783] T ee2e79e677f643409e3c8dd49b9517cf P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:06.573033 13869 tablet_copy_client.cc:647] T ee2e79e677f643409e3c8dd49b9517cf P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:06.573714 13868 tablet_copy_client.cc:1002] T 8cf1917de461431d98c5155db3eb2312 P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Injecting latency into file download: 1500
I20240502 07:59:06.573833 13873 tablet_copy_client.cc:1002] T ee2e79e677f643409e3c8dd49b9517cf P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Injecting latency into file download: 1500
I20240502 07:59:07.404165 13587 tablet_copy_client.cc:523] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:07.405349 13587 tablet_bootstrap.cc:492] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8eb9945f989d4be6b51e1531a1365350: Bootstrap starting.
I20240502 07:59:07.422169 13587 tablet_bootstrap.cc:492] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8eb9945f989d4be6b51e1531a1365350: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:07.422376 13587 tablet_bootstrap.cc:492] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8eb9945f989d4be6b51e1531a1365350: Bootstrap complete.
I20240502 07:59:07.422497 13587 ts_tablet_manager.cc:1367] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8eb9945f989d4be6b51e1531a1365350: Time spent bootstrapping tablet: real 0.017s user 0.015s sys 0.000s
I20240502 07:59:07.422794 13587 raft_consensus.cc:348] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8eb9945f989d4be6b51e1531a1365350 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } }
I20240502 07:59:07.422904 13587 raft_consensus.cc:729] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8eb9945f989d4be6b51e1531a1365350 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 8eb9945f989d4be6b51e1531a1365350, State: Initialized, Role: LEARNER
I20240502 07:59:07.422974 13587 consensus_queue.cc:260] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8eb9945f989d4be6b51e1531a1365350 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } }
I20240502 07:59:07.423249 13587 ts_tablet_manager.cc:1398] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8eb9945f989d4be6b51e1531a1365350: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240502 07:59:07.423667 9666 tablet_copy_service.cc:342] P 8337af1943564c0dad18b45a597b805a: Request end of tablet copy session 8eb9945f989d4be6b51e1531a1365350-427c037fa09c4b94b7bc6c71bdb61c2f received from {username='slave'} at 127.3.139.70:48793
I20240502 07:59:07.423759 9666 tablet_copy_service.cc:433] P 8337af1943564c0dad18b45a597b805a: ending tablet copy session 8eb9945f989d4be6b51e1531a1365350-427c037fa09c4b94b7bc6c71bdb61c2f on tablet 427c037fa09c4b94b7bc6c71bdb61c2f with peer 8eb9945f989d4be6b51e1531a1365350
I20240502 07:59:07.632340 13691 tablet_copy_client.cc:523] T 0e4efe560ef74763b80b70665c11d7f1 P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:07.633539 13691 tablet_bootstrap.cc:492] T 0e4efe560ef74763b80b70665c11d7f1 P 98bef2e94ade4763a66c1ad68259744a: Bootstrap starting.
I20240502 07:59:07.650471 13691 tablet_bootstrap.cc:492] T 0e4efe560ef74763b80b70665c11d7f1 P 98bef2e94ade4763a66c1ad68259744a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:07.650691 13691 tablet_bootstrap.cc:492] T 0e4efe560ef74763b80b70665c11d7f1 P 98bef2e94ade4763a66c1ad68259744a: Bootstrap complete.
I20240502 07:59:07.650820 13691 ts_tablet_manager.cc:1367] T 0e4efe560ef74763b80b70665c11d7f1 P 98bef2e94ade4763a66c1ad68259744a: Time spent bootstrapping tablet: real 0.017s user 0.015s sys 0.000s
I20240502 07:59:07.651188 13691 raft_consensus.cc:348] T 0e4efe560ef74763b80b70665c11d7f1 P 98bef2e94ade4763a66c1ad68259744a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } }
I20240502 07:59:07.651293 13691 raft_consensus.cc:729] T 0e4efe560ef74763b80b70665c11d7f1 P 98bef2e94ade4763a66c1ad68259744a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 98bef2e94ade4763a66c1ad68259744a, State: Initialized, Role: LEARNER
I20240502 07:59:07.651371 13691 consensus_queue.cc:260] T 0e4efe560ef74763b80b70665c11d7f1 P 98bef2e94ade4763a66c1ad68259744a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } }
I20240502 07:59:07.651822 13691 ts_tablet_manager.cc:1398] T 0e4efe560ef74763b80b70665c11d7f1 P 98bef2e94ade4763a66c1ad68259744a: Time spent starting tablet: real 0.001s user 0.005s sys 0.000s
I20240502 07:59:07.652354 9797 tablet_copy_service.cc:342] P 1472c64688914149a318c7b8850a3df3: Request end of tablet copy session 98bef2e94ade4763a66c1ad68259744a-0e4efe560ef74763b80b70665c11d7f1 received from {username='slave'} at 127.3.139.71:56869
I20240502 07:59:07.652438 9797 tablet_copy_service.cc:433] P 1472c64688914149a318c7b8850a3df3: ending tablet copy session 98bef2e94ade4763a66c1ad68259744a-0e4efe560ef74763b80b70665c11d7f1 on tablet 0e4efe560ef74763b80b70665c11d7f1 with peer 98bef2e94ade4763a66c1ad68259744a
I20240502 07:59:07.719503 13697 tablet_copy_client.cc:523] T f7335ac0209c4e28b6751bb01d34d665 P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:07.720762 13697 tablet_bootstrap.cc:492] T f7335ac0209c4e28b6751bb01d34d665 P 51e82953ed2247fa9ff24638eab3508a: Bootstrap starting.
I20240502 07:59:07.739910 13697 tablet_bootstrap.cc:492] T f7335ac0209c4e28b6751bb01d34d665 P 51e82953ed2247fa9ff24638eab3508a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:07.740192 13697 tablet_bootstrap.cc:492] T f7335ac0209c4e28b6751bb01d34d665 P 51e82953ed2247fa9ff24638eab3508a: Bootstrap complete.
I20240502 07:59:07.740365 13697 ts_tablet_manager.cc:1367] T f7335ac0209c4e28b6751bb01d34d665 P 51e82953ed2247fa9ff24638eab3508a: Time spent bootstrapping tablet: real 0.020s user 0.020s sys 0.000s
I20240502 07:59:07.740684 13697 raft_consensus.cc:348] T f7335ac0209c4e28b6751bb01d34d665 P 51e82953ed2247fa9ff24638eab3508a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } }
I20240502 07:59:07.740782 13697 raft_consensus.cc:729] T f7335ac0209c4e28b6751bb01d34d665 P 51e82953ed2247fa9ff24638eab3508a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 51e82953ed2247fa9ff24638eab3508a, State: Initialized, Role: LEARNER
I20240502 07:59:07.740873 13697 consensus_queue.cc:260] T f7335ac0209c4e28b6751bb01d34d665 P 51e82953ed2247fa9ff24638eab3508a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } }
I20240502 07:59:07.741185 13697 ts_tablet_manager.cc:1398] T f7335ac0209c4e28b6751bb01d34d665 P 51e82953ed2247fa9ff24638eab3508a: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 07:59:07.741647 9666 tablet_copy_service.cc:342] P 8337af1943564c0dad18b45a597b805a: Request end of tablet copy session 51e82953ed2247fa9ff24638eab3508a-f7335ac0209c4e28b6751bb01d34d665 received from {username='slave'} at 127.3.139.69:35563
I20240502 07:59:07.741737 9666 tablet_copy_service.cc:433] P 8337af1943564c0dad18b45a597b805a: ending tablet copy session 51e82953ed2247fa9ff24638eab3508a-f7335ac0209c4e28b6751bb01d34d665 on tablet f7335ac0209c4e28b6751bb01d34d665 with peer 51e82953ed2247fa9ff24638eab3508a
I20240502 07:59:07.805884 13782 tablet_copy_client.cc:523] T d37b8a7f016647a7a03baedc37ed5419 P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:07.807322 13782 tablet_bootstrap.cc:492] T d37b8a7f016647a7a03baedc37ed5419 P a95f1d3af33f4c879fd290ca917d6329: Bootstrap starting.
I20240502 07:59:07.824703 13782 tablet_bootstrap.cc:492] T d37b8a7f016647a7a03baedc37ed5419 P a95f1d3af33f4c879fd290ca917d6329: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:07.824894 13782 tablet_bootstrap.cc:492] T d37b8a7f016647a7a03baedc37ed5419 P a95f1d3af33f4c879fd290ca917d6329: Bootstrap complete.
I20240502 07:59:07.825016 13782 ts_tablet_manager.cc:1367] T d37b8a7f016647a7a03baedc37ed5419 P a95f1d3af33f4c879fd290ca917d6329: Time spent bootstrapping tablet: real 0.018s user 0.016s sys 0.000s
I20240502 07:59:07.825347 13782 raft_consensus.cc:348] T d37b8a7f016647a7a03baedc37ed5419 P a95f1d3af33f4c879fd290ca917d6329 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } }
I20240502 07:59:07.825444 13782 raft_consensus.cc:729] T d37b8a7f016647a7a03baedc37ed5419 P a95f1d3af33f4c879fd290ca917d6329 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a95f1d3af33f4c879fd290ca917d6329, State: Initialized, Role: LEARNER
I20240502 07:59:07.825524 13782 consensus_queue.cc:260] T d37b8a7f016647a7a03baedc37ed5419 P a95f1d3af33f4c879fd290ca917d6329 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } }
I20240502 07:59:07.825804 13782 ts_tablet_manager.cc:1398] T d37b8a7f016647a7a03baedc37ed5419 P a95f1d3af33f4c879fd290ca917d6329: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240502 07:59:07.826335 9797 tablet_copy_service.cc:342] P 1472c64688914149a318c7b8850a3df3: Request end of tablet copy session a95f1d3af33f4c879fd290ca917d6329-d37b8a7f016647a7a03baedc37ed5419 received from {username='slave'} at 127.3.139.72:51819
I20240502 07:59:07.826421 9797 tablet_copy_service.cc:433] P 1472c64688914149a318c7b8850a3df3: ending tablet copy session a95f1d3af33f4c879fd290ca917d6329-d37b8a7f016647a7a03baedc37ed5419 on tablet d37b8a7f016647a7a03baedc37ed5419 with peer a95f1d3af33f4c879fd290ca917d6329
I20240502 07:59:07.829002 13803 tablet_copy_client.cc:523] T 70d4db3c3291490db58678522d356688 P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:07.830078 13803 tablet_bootstrap.cc:492] T 70d4db3c3291490db58678522d356688 P 8eb9945f989d4be6b51e1531a1365350: Bootstrap starting.
I20240502 07:59:07.834585 13808 tablet_copy_client.cc:523] T 12e3bfe638d249c0934feda41b8a8876 P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:07.835745 13808 tablet_bootstrap.cc:492] T 12e3bfe638d249c0934feda41b8a8876 P 8eb9945f989d4be6b51e1531a1365350: Bootstrap starting.
I20240502 07:59:07.843453 11389 raft_consensus.cc:1203] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8eb9945f989d4be6b51e1531a1365350 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:07.848203 13803 tablet_bootstrap.cc:492] T 70d4db3c3291490db58678522d356688 P 8eb9945f989d4be6b51e1531a1365350: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:07.848438 13803 tablet_bootstrap.cc:492] T 70d4db3c3291490db58678522d356688 P 8eb9945f989d4be6b51e1531a1365350: Bootstrap complete.
I20240502 07:59:07.848579 13803 ts_tablet_manager.cc:1367] T 70d4db3c3291490db58678522d356688 P 8eb9945f989d4be6b51e1531a1365350: Time spent bootstrapping tablet: real 0.019s user 0.014s sys 0.003s
I20240502 07:59:07.848902 13803 raft_consensus.cc:348] T 70d4db3c3291490db58678522d356688 P 8eb9945f989d4be6b51e1531a1365350 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } }
I20240502 07:59:07.848997 13803 raft_consensus.cc:729] T 70d4db3c3291490db58678522d356688 P 8eb9945f989d4be6b51e1531a1365350 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 8eb9945f989d4be6b51e1531a1365350, State: Initialized, Role: LEARNER
I20240502 07:59:07.849082 13803 consensus_queue.cc:260] T 70d4db3c3291490db58678522d356688 P 8eb9945f989d4be6b51e1531a1365350 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } }
I20240502 07:59:07.849313 13803 ts_tablet_manager.cc:1398] T 70d4db3c3291490db58678522d356688 P 8eb9945f989d4be6b51e1531a1365350: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:07.849783 9797 tablet_copy_service.cc:342] P 1472c64688914149a318c7b8850a3df3: Request end of tablet copy session 8eb9945f989d4be6b51e1531a1365350-70d4db3c3291490db58678522d356688 received from {username='slave'} at 127.3.139.70:41973
I20240502 07:59:07.849871 9797 tablet_copy_service.cc:433] P 1472c64688914149a318c7b8850a3df3: ending tablet copy session 8eb9945f989d4be6b51e1531a1365350-70d4db3c3291490db58678522d356688 on tablet 70d4db3c3291490db58678522d356688 with peer 8eb9945f989d4be6b51e1531a1365350
I20240502 07:59:07.854624 13808 tablet_bootstrap.cc:492] T 12e3bfe638d249c0934feda41b8a8876 P 8eb9945f989d4be6b51e1531a1365350: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:07.854821 13808 tablet_bootstrap.cc:492] T 12e3bfe638d249c0934feda41b8a8876 P 8eb9945f989d4be6b51e1531a1365350: Bootstrap complete.
I20240502 07:59:07.854962 13808 ts_tablet_manager.cc:1367] T 12e3bfe638d249c0934feda41b8a8876 P 8eb9945f989d4be6b51e1531a1365350: Time spent bootstrapping tablet: real 0.019s user 0.012s sys 0.004s
I20240502 07:59:07.855320 13808 raft_consensus.cc:348] T 12e3bfe638d249c0934feda41b8a8876 P 8eb9945f989d4be6b51e1531a1365350 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } }
I20240502 07:59:07.855409 13808 raft_consensus.cc:729] T 12e3bfe638d249c0934feda41b8a8876 P 8eb9945f989d4be6b51e1531a1365350 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 8eb9945f989d4be6b51e1531a1365350, State: Initialized, Role: LEARNER
I20240502 07:59:07.855489 13808 consensus_queue.cc:260] T 12e3bfe638d249c0934feda41b8a8876 P 8eb9945f989d4be6b51e1531a1365350 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } }
I20240502 07:59:07.855701 13808 ts_tablet_manager.cc:1398] T 12e3bfe638d249c0934feda41b8a8876 P 8eb9945f989d4be6b51e1531a1365350: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240502 07:59:07.856104 9666 tablet_copy_service.cc:342] P 8337af1943564c0dad18b45a597b805a: Request end of tablet copy session 8eb9945f989d4be6b51e1531a1365350-12e3bfe638d249c0934feda41b8a8876 received from {username='slave'} at 127.3.139.70:48793
I20240502 07:59:07.856173 9666 tablet_copy_service.cc:433] P 8337af1943564c0dad18b45a597b805a: ending tablet copy session 8eb9945f989d4be6b51e1531a1365350-12e3bfe638d249c0934feda41b8a8876 on tablet 12e3bfe638d249c0934feda41b8a8876 with peer 8eb9945f989d4be6b51e1531a1365350
I20240502 07:59:07.872853 11389 raft_consensus.cc:1203] T 12e3bfe638d249c0934feda41b8a8876 P 8eb9945f989d4be6b51e1531a1365350 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:07.940690 13813 tablet_copy_client.cc:523] T 265eca6fff644d83857ed48a04864a58 P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:07.941892 13813 tablet_bootstrap.cc:492] T 265eca6fff644d83857ed48a04864a58 P a95f1d3af33f4c879fd290ca917d6329: Bootstrap starting.
I20240502 07:59:07.959300 13813 tablet_bootstrap.cc:492] T 265eca6fff644d83857ed48a04864a58 P a95f1d3af33f4c879fd290ca917d6329: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:07.959494 13813 tablet_bootstrap.cc:492] T 265eca6fff644d83857ed48a04864a58 P a95f1d3af33f4c879fd290ca917d6329: Bootstrap complete.
I20240502 07:59:07.959611 13813 ts_tablet_manager.cc:1367] T 265eca6fff644d83857ed48a04864a58 P a95f1d3af33f4c879fd290ca917d6329: Time spent bootstrapping tablet: real 0.018s user 0.018s sys 0.000s
I20240502 07:59:07.959925 13813 raft_consensus.cc:348] T 265eca6fff644d83857ed48a04864a58 P a95f1d3af33f4c879fd290ca917d6329 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } }
I20240502 07:59:07.960021 13813 raft_consensus.cc:729] T 265eca6fff644d83857ed48a04864a58 P a95f1d3af33f4c879fd290ca917d6329 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a95f1d3af33f4c879fd290ca917d6329, State: Initialized, Role: LEARNER
I20240502 07:59:07.960094 13813 consensus_queue.cc:260] T 265eca6fff644d83857ed48a04864a58 P a95f1d3af33f4c879fd290ca917d6329 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } }
I20240502 07:59:07.960351 13813 ts_tablet_manager.cc:1398] T 265eca6fff644d83857ed48a04864a58 P a95f1d3af33f4c879fd290ca917d6329: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:07.960814 9534 tablet_copy_service.cc:342] P a201db721bda4dc3a4422dc8fa9bf72e: Request end of tablet copy session a95f1d3af33f4c879fd290ca917d6329-265eca6fff644d83857ed48a04864a58 received from {username='slave'} at 127.3.139.72:38421
I20240502 07:59:07.960886 9534 tablet_copy_service.cc:433] P a201db721bda4dc3a4422dc8fa9bf72e: ending tablet copy session a95f1d3af33f4c879fd290ca917d6329-265eca6fff644d83857ed48a04864a58 on tablet 265eca6fff644d83857ed48a04864a58 with peer a95f1d3af33f4c879fd290ca917d6329
I20240502 07:59:08.033519 11520 raft_consensus.cc:1203] T 0e4efe560ef74763b80b70665c11d7f1 P 98bef2e94ade4763a66c1ad68259744a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:08.073990 13864 tablet_copy_client.cc:523] T 8cf1917de461431d98c5155db3eb2312 P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:08.074103 13869 tablet_copy_client.cc:523] T ee2e79e677f643409e3c8dd49b9517cf P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:08.075309 13869 tablet_bootstrap.cc:492] T ee2e79e677f643409e3c8dd49b9517cf P a95f1d3af33f4c879fd290ca917d6329: Bootstrap starting.
I20240502 07:59:08.075310 13864 tablet_bootstrap.cc:492] T 8cf1917de461431d98c5155db3eb2312 P a95f1d3af33f4c879fd290ca917d6329: Bootstrap starting.
I20240502 07:59:08.092752 13869 tablet_bootstrap.cc:492] T ee2e79e677f643409e3c8dd49b9517cf P a95f1d3af33f4c879fd290ca917d6329: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:08.092953 13869 tablet_bootstrap.cc:492] T ee2e79e677f643409e3c8dd49b9517cf P a95f1d3af33f4c879fd290ca917d6329: Bootstrap complete.
I20240502 07:59:08.093091 13869 ts_tablet_manager.cc:1367] T ee2e79e677f643409e3c8dd49b9517cf P a95f1d3af33f4c879fd290ca917d6329: Time spent bootstrapping tablet: real 0.018s user 0.016s sys 0.000s
I20240502 07:59:08.093426 13869 raft_consensus.cc:348] T ee2e79e677f643409e3c8dd49b9517cf P a95f1d3af33f4c879fd290ca917d6329 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } }
I20240502 07:59:08.093524 13869 raft_consensus.cc:729] T ee2e79e677f643409e3c8dd49b9517cf P a95f1d3af33f4c879fd290ca917d6329 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a95f1d3af33f4c879fd290ca917d6329, State: Initialized, Role: LEARNER
I20240502 07:59:08.093609 13869 consensus_queue.cc:260] T ee2e79e677f643409e3c8dd49b9517cf P a95f1d3af33f4c879fd290ca917d6329 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } }
I20240502 07:59:08.094197 13869 ts_tablet_manager.cc:1398] T ee2e79e677f643409e3c8dd49b9517cf P a95f1d3af33f4c879fd290ca917d6329: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240502 07:59:08.094688 9534 tablet_copy_service.cc:342] P a201db721bda4dc3a4422dc8fa9bf72e: Request end of tablet copy session a95f1d3af33f4c879fd290ca917d6329-ee2e79e677f643409e3c8dd49b9517cf received from {username='slave'} at 127.3.139.72:38421
I20240502 07:59:08.094779 9534 tablet_copy_service.cc:433] P a201db721bda4dc3a4422dc8fa9bf72e: ending tablet copy session a95f1d3af33f4c879fd290ca917d6329-ee2e79e677f643409e3c8dd49b9517cf on tablet ee2e79e677f643409e3c8dd49b9517cf with peer a95f1d3af33f4c879fd290ca917d6329
I20240502 07:59:08.098242 13864 tablet_bootstrap.cc:492] T 8cf1917de461431d98c5155db3eb2312 P a95f1d3af33f4c879fd290ca917d6329: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:08.098443 13864 tablet_bootstrap.cc:492] T 8cf1917de461431d98c5155db3eb2312 P a95f1d3af33f4c879fd290ca917d6329: Bootstrap complete.
I20240502 07:59:08.098592 13864 ts_tablet_manager.cc:1367] T 8cf1917de461431d98c5155db3eb2312 P a95f1d3af33f4c879fd290ca917d6329: Time spent bootstrapping tablet: real 0.023s user 0.017s sys 0.000s
I20240502 07:59:08.098929 13864 raft_consensus.cc:348] T 8cf1917de461431d98c5155db3eb2312 P a95f1d3af33f4c879fd290ca917d6329 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } }
I20240502 07:59:08.099040 13864 raft_consensus.cc:729] T 8cf1917de461431d98c5155db3eb2312 P a95f1d3af33f4c879fd290ca917d6329 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a95f1d3af33f4c879fd290ca917d6329, State: Initialized, Role: LEARNER
I20240502 07:59:08.099166 13864 consensus_queue.cc:260] T 8cf1917de461431d98c5155db3eb2312 P a95f1d3af33f4c879fd290ca917d6329 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } }
I20240502 07:59:08.099401 13864 ts_tablet_manager.cc:1398] T 8cf1917de461431d98c5155db3eb2312 P a95f1d3af33f4c879fd290ca917d6329: Time spent starting tablet: real 0.001s user 0.000s sys 0.003s
I20240502 07:59:08.099870 9534 tablet_copy_service.cc:342] P a201db721bda4dc3a4422dc8fa9bf72e: Request end of tablet copy session a95f1d3af33f4c879fd290ca917d6329-8cf1917de461431d98c5155db3eb2312 received from {username='slave'} at 127.3.139.72:38421
I20240502 07:59:08.099937 9534 tablet_copy_service.cc:433] P a201db721bda4dc3a4422dc8fa9bf72e: ending tablet copy session a95f1d3af33f4c879fd290ca917d6329-8cf1917de461431d98c5155db3eb2312 on tablet 8cf1917de461431d98c5155db3eb2312 with peer a95f1d3af33f4c879fd290ca917d6329
I20240502 07:59:08.104292 11259 raft_consensus.cc:1203] T f7335ac0209c4e28b6751bb01d34d665 P 51e82953ed2247fa9ff24638eab3508a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:08.108958 11655 raft_consensus.cc:1203] T 8cf1917de461431d98c5155db3eb2312 P a95f1d3af33f4c879fd290ca917d6329 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:08.181540 11655 raft_consensus.cc:1203] T d37b8a7f016647a7a03baedc37ed5419 P a95f1d3af33f4c879fd290ca917d6329 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:08.284869 11389 raft_consensus.cc:1203] T 70d4db3c3291490db58678522d356688 P 8eb9945f989d4be6b51e1531a1365350 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:08.364261 11655 raft_consensus.cc:1203] T 265eca6fff644d83857ed48a04864a58 P a95f1d3af33f4c879fd290ca917d6329 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:08.432807 14205 raft_consensus.cc:1050] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3: attempting to promote NON_VOTER 98bef2e94ade4763a66c1ad68259744a to VOTER
I20240502 07:59:08.433223 14205 consensus_queue.cc:237] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } }
I20240502 07:59:08.434502 9514 raft_consensus.cc:1261] T 0e4efe560ef74763b80b70665c11d7f1 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:08.434630 11520 raft_consensus.cc:1261] T 0e4efe560ef74763b80b70665c11d7f1 P 98bef2e94ade4763a66c1ad68259744a [term 1 LEARNER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:08.434626 9906 raft_consensus.cc:1261] T 0e4efe560ef74763b80b70665c11d7f1 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:08.435065 14204 consensus_queue.cc:1035] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:08.435288 14207 consensus_queue.cc:1035] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:08.435470 14205 consensus_queue.cc:1035] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:08.437721 14205 raft_consensus.cc:2932] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:08.438257 9906 raft_consensus.cc:2932] T 0e4efe560ef74763b80b70665c11d7f1 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:08.438446 11520 raft_consensus.cc:2932] T 0e4efe560ef74763b80b70665c11d7f1 P 98bef2e94ade4763a66c1ad68259744a [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:08.440922 9365 catalog_manager.cc:5484] T 0e4efe560ef74763b80b70665c11d7f1 P 6cc5b9962ed24441bcba0c07b9f5438e reported cstate change: config changed from index 4 to 5, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1472c64688914149a318c7b8850a3df3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:08.441159 9514 raft_consensus.cc:2932] T 0e4efe560ef74763b80b70665c11d7f1 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:08.444857 9776 consensus_queue.cc:237] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } }
I20240502 07:59:08.445897 11520 raft_consensus.cc:1261] T 0e4efe560ef74763b80b70665c11d7f1 P 98bef2e94ade4763a66c1ad68259744a [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:08.446349 9906 raft_consensus.cc:1261] T 0e4efe560ef74763b80b70665c11d7f1 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:08.446362 14205 consensus_queue.cc:1035] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:08.446846 14205 consensus_queue.cc:1035] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:08.447958 14204 raft_consensus.cc:2932] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:08.448519 9906 raft_consensus.cc:2932] T 0e4efe560ef74763b80b70665c11d7f1 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:08.448556 11520 raft_consensus.cc:2932] T 0e4efe560ef74763b80b70665c11d7f1 P 98bef2e94ade4763a66c1ad68259744a [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:08.449818 9360 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 0e4efe560ef74763b80b70665c11d7f1 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:08.450729 9367 catalog_manager.cc:5484] T 0e4efe560ef74763b80b70665c11d7f1 P 6cc5b9962ed24441bcba0c07b9f5438e reported cstate change: config changed from index 5 to 6, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New cstate: current_term: 1 leader_uuid: "1472c64688914149a318c7b8850a3df3" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:08.452821 9493 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e4efe560ef74763b80b70665c11d7f1 with delete_type TABLET_DATA_TOMBSTONED (TS a201db721bda4dc3a4422dc8fa9bf72e not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:41730
I20240502 07:59:08.453117 14215 tablet_replica.cc:330] T 0e4efe560ef74763b80b70665c11d7f1 P a201db721bda4dc3a4422dc8fa9bf72e: stopping tablet replica
I20240502 07:59:08.453250 14215 raft_consensus.cc:2229] T 0e4efe560ef74763b80b70665c11d7f1 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:08.453393 14215 raft_consensus.cc:2258] T 0e4efe560ef74763b80b70665c11d7f1 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:08.453931 14215 ts_tablet_manager.cc:1875] T 0e4efe560ef74763b80b70665c11d7f1 P a201db721bda4dc3a4422dc8fa9bf72e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:08.455783 14215 ts_tablet_manager.cc:1888] T 0e4efe560ef74763b80b70665c11d7f1 P a201db721bda4dc3a4422dc8fa9bf72e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 07:59:08.455849 14215 log.cc:1196] T 0e4efe560ef74763b80b70665c11d7f1 P a201db721bda4dc3a4422dc8fa9bf72e: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal/wals/0e4efe560ef74763b80b70665c11d7f1
I20240502 07:59:08.456254 9360 catalog_manager.cc:4830] TS a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551): tablet 0e4efe560ef74763b80b70665c11d7f1 (table rebalance_test_table_8 [id=6aecdf32373e4c9db62c5a4844dc38ca]) successfully deleted
I20240502 07:59:08.459874 14217 raft_consensus.cc:1050] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a: attempting to promote NON_VOTER 8eb9945f989d4be6b51e1531a1365350 to VOTER
I20240502 07:59:08.460314 14217 consensus_queue.cc:237] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } }
I20240502 07:59:08.461494 11389 raft_consensus.cc:1261] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8eb9945f989d4be6b51e1531a1365350 [term 1 LEARNER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:08.461505 9906 raft_consensus.cc:1261] T 427c037fa09c4b94b7bc6c71bdb61c2f P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:08.461596 9776 raft_consensus.cc:1261] T 427c037fa09c4b94b7bc6c71bdb61c2f P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:08.461961 14219 consensus_queue.cc:1035] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:08.462143 13328 consensus_queue.cc:1035] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:08.462273 14216 consensus_queue.cc:1035] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:08.463960 13328 raft_consensus.cc:2932] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:08.464447 9776 raft_consensus.cc:2932] T 427c037fa09c4b94b7bc6c71bdb61c2f P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:08.464774 9906 raft_consensus.cc:2932] T 427c037fa09c4b94b7bc6c71bdb61c2f P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:08.466432 11389 raft_consensus.cc:2932] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:08.466665 9370 catalog_manager.cc:5484] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a reported cstate change: config changed from index 4 to 5, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:08.491746 14219 raft_consensus.cc:1050] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a: attempting to promote NON_VOTER 8eb9945f989d4be6b51e1531a1365350 to VOTER
I20240502 07:59:08.492195 14219 consensus_queue.cc:237] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } }
I20240502 07:59:08.493497 9514 raft_consensus.cc:1261] T 12e3bfe638d249c0934feda41b8a8876 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:08.493497 9906 raft_consensus.cc:1261] T 12e3bfe638d249c0934feda41b8a8876 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:08.493830 11389 raft_consensus.cc:1261] T 12e3bfe638d249c0934feda41b8a8876 P 8eb9945f989d4be6b51e1531a1365350 [term 1 LEARNER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:08.493924 14216 consensus_queue.cc:1035] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:08.494187 13328 consensus_queue.cc:1035] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:08.494498 14217 consensus_queue.cc:1035] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:08.496325 14217 raft_consensus.cc:2932] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:08.496766 9906 raft_consensus.cc:2932] T 12e3bfe638d249c0934feda41b8a8876 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:08.498261 11389 raft_consensus.cc:2932] T 12e3bfe638d249c0934feda41b8a8876 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:08.498333 9514 raft_consensus.cc:2932] T 12e3bfe638d249c0934feda41b8a8876 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:08.499004 9365 catalog_manager.cc:5484] T 12e3bfe638d249c0934feda41b8a8876 P 6cc5b9962ed24441bcba0c07b9f5438e reported cstate change: config changed from index 4 to 5, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:08.502907 9644 consensus_queue.cc:237] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } }
I20240502 07:59:08.504123 11389 raft_consensus.cc:1261] T 12e3bfe638d249c0934feda41b8a8876 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:08.504421 13328 consensus_queue.cc:1035] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:08.504536 9514 raft_consensus.cc:1261] T 12e3bfe638d249c0934feda41b8a8876 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:08.504868 13328 consensus_queue.cc:1035] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:08.506949 14216 raft_consensus.cc:2932] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:08.507388 11389 raft_consensus.cc:2932] T 12e3bfe638d249c0934feda41b8a8876 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:08.508347 9358 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 12e3bfe638d249c0934feda41b8a8876 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:08.509135 9514 raft_consensus.cc:2932] T 12e3bfe638d249c0934feda41b8a8876 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:08.509521 9370 catalog_manager.cc:5484] T 12e3bfe638d249c0934feda41b8a8876 P 8eb9945f989d4be6b51e1531a1365350 reported cstate change: config changed from index 5 to 6, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:08.512980 9886 tablet_service.cc:1514] Processing DeleteTablet for tablet 12e3bfe638d249c0934feda41b8a8876 with delete_type TABLET_DATA_TOMBSTONED (TS 6cc5b9962ed24441bcba0c07b9f5438e not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:41940
I20240502 07:59:08.513284 14256 tablet_replica.cc:330] T 12e3bfe638d249c0934feda41b8a8876 P 6cc5b9962ed24441bcba0c07b9f5438e: stopping tablet replica
I20240502 07:59:08.513418 14256 raft_consensus.cc:2229] T 12e3bfe638d249c0934feda41b8a8876 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:08.513576 14256 raft_consensus.cc:2258] T 12e3bfe638d249c0934feda41b8a8876 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:08.514231 14256 ts_tablet_manager.cc:1875] T 12e3bfe638d249c0934feda41b8a8876 P 6cc5b9962ed24441bcba0c07b9f5438e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:08.516218 14256 ts_tablet_manager.cc:1888] T 12e3bfe638d249c0934feda41b8a8876 P 6cc5b9962ed24441bcba0c07b9f5438e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 07:59:08.516326 14256 log.cc:1196] T 12e3bfe638d249c0934feda41b8a8876 P 6cc5b9962ed24441bcba0c07b9f5438e: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal/wals/12e3bfe638d249c0934feda41b8a8876
I20240502 07:59:08.516803 9358 catalog_manager.cc:4830] TS 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379): tablet 12e3bfe638d249c0934feda41b8a8876 (table rebalance_test_table_5 [id=51b1431ec18d4bb8b4d5ae1909ae1322]) successfully deleted
I20240502 07:59:08.535379 9644 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "427c037fa09c4b94b7bc6c71bdb61c2f"
dest_uuid: "8337af1943564c0dad18b45a597b805a"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:37508
I20240502 07:59:08.535511 9644 raft_consensus.cc:595] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Received request to transfer leadership
I20240502 07:59:08.573498 9644 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "427c037fa09c4b94b7bc6c71bdb61c2f"
dest_uuid: "8337af1943564c0dad18b45a597b805a"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:37548
I20240502 07:59:08.573606 9644 raft_consensus.cc:595] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Received request to transfer leadership
I20240502 07:59:08.620870 14319 raft_consensus.cc:1050] T 8cf1917de461431d98c5155db3eb2312 P a201db721bda4dc3a4422dc8fa9bf72e: attempting to promote NON_VOTER a95f1d3af33f4c879fd290ca917d6329 to VOTER
I20240502 07:59:08.621299 14319 consensus_queue.cc:237] T 8cf1917de461431d98c5155db3eb2312 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } }
I20240502 07:59:08.622574 11655 raft_consensus.cc:1261] T 8cf1917de461431d98c5155db3eb2312 P a95f1d3af33f4c879fd290ca917d6329 [term 1 LEARNER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:08.622655 9644 raft_consensus.cc:1261] T 8cf1917de461431d98c5155db3eb2312 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:08.622665 9777 raft_consensus.cc:1261] T 8cf1917de461431d98c5155db3eb2312 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:08.622958 14318 consensus_queue.cc:1035] T 8cf1917de461431d98c5155db3eb2312 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:08.623178 14319 consensus_queue.cc:1035] T 8cf1917de461431d98c5155db3eb2312 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:08.623332 14321 consensus_queue.cc:1035] T 8cf1917de461431d98c5155db3eb2312 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:08.625396 14318 raft_consensus.cc:2932] T 8cf1917de461431d98c5155db3eb2312 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:08.625883 11655 raft_consensus.cc:2932] T 8cf1917de461431d98c5155db3eb2312 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:08.626031 9644 raft_consensus.cc:2932] T 8cf1917de461431d98c5155db3eb2312 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:08.627225 9777 raft_consensus.cc:2932] T 8cf1917de461431d98c5155db3eb2312 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:08.628247 9369 catalog_manager.cc:5484] T 8cf1917de461431d98c5155db3eb2312 P 8337af1943564c0dad18b45a597b805a reported cstate change: config changed from index 4 to 5, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:08.631808 9514 consensus_queue.cc:237] T 8cf1917de461431d98c5155db3eb2312 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } }
I20240502 07:59:08.632691 9777 raft_consensus.cc:1261] T 8cf1917de461431d98c5155db3eb2312 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:08.632750 11655 raft_consensus.cc:1261] T 8cf1917de461431d98c5155db3eb2312 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:08.633035 14318 consensus_queue.cc:1035] T 8cf1917de461431d98c5155db3eb2312 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:08.633201 14321 consensus_queue.cc:1035] T 8cf1917de461431d98c5155db3eb2312 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:08.634763 14318 raft_consensus.cc:2932] T 8cf1917de461431d98c5155db3eb2312 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:08.634936 11655 raft_consensus.cc:2932] T 8cf1917de461431d98c5155db3eb2312 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:08.635102 9777 raft_consensus.cc:2932] T 8cf1917de461431d98c5155db3eb2312 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:08.636175 9360 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 8cf1917de461431d98c5155db3eb2312 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:08.637269 9370 catalog_manager.cc:5484] T 8cf1917de461431d98c5155db3eb2312 P a201db721bda4dc3a4422dc8fa9bf72e reported cstate change: config changed from index 5 to 6, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:08.639351 9624 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cf1917de461431d98c5155db3eb2312 with delete_type TABLET_DATA_TOMBSTONED (TS 8337af1943564c0dad18b45a597b805a not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:53862
I20240502 07:59:08.639736 14325 tablet_replica.cc:330] T 8cf1917de461431d98c5155db3eb2312 P 8337af1943564c0dad18b45a597b805a: stopping tablet replica
I20240502 07:59:08.639890 14325 raft_consensus.cc:2229] T 8cf1917de461431d98c5155db3eb2312 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:08.640067 14325 raft_consensus.cc:2258] T 8cf1917de461431d98c5155db3eb2312 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:08.640679 14325 ts_tablet_manager.cc:1875] T 8cf1917de461431d98c5155db3eb2312 P 8337af1943564c0dad18b45a597b805a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:08.642230 14325 ts_tablet_manager.cc:1888] T 8cf1917de461431d98c5155db3eb2312 P 8337af1943564c0dad18b45a597b805a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 07:59:08.642288 14325 log.cc:1196] T 8cf1917de461431d98c5155db3eb2312 P 8337af1943564c0dad18b45a597b805a: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal/wals/8cf1917de461431d98c5155db3eb2312
I20240502 07:59:08.642680 9358 catalog_manager.cc:4830] TS 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667): tablet 8cf1917de461431d98c5155db3eb2312 (table rebalance_test_table_6 [id=af6c4a01744d4fcfbdf53cfc8d19500b]) successfully deleted
I20240502 07:59:08.649713 11655 raft_consensus.cc:1203] T ee2e79e677f643409e3c8dd49b9517cf P a95f1d3af33f4c879fd290ca917d6329 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:08.686357 14219 raft_consensus.cc:1050] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a: attempting to promote NON_VOTER 51e82953ed2247fa9ff24638eab3508a to VOTER
I20240502 07:59:08.686688 14219 consensus_queue.cc:237] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } }
I20240502 07:59:08.687916 9514 raft_consensus.cc:1261] T f7335ac0209c4e28b6751bb01d34d665 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:08.687939 9906 raft_consensus.cc:1261] T f7335ac0209c4e28b6751bb01d34d665 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:08.687945 11259 raft_consensus.cc:1261] T f7335ac0209c4e28b6751bb01d34d665 P 51e82953ed2247fa9ff24638eab3508a [term 1 LEARNER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:08.688306 14217 consensus_queue.cc:1035] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:08.688482 13328 consensus_queue.cc:1035] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:08.688611 14219 consensus_queue.cc:1035] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:08.690508 14219 raft_consensus.cc:2932] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:08.691180 9514 raft_consensus.cc:2932] T f7335ac0209c4e28b6751bb01d34d665 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:08.692771 9906 raft_consensus.cc:2932] T f7335ac0209c4e28b6751bb01d34d665 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:08.693056 9367 catalog_manager.cc:5484] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a reported cstate change: config changed from index 4 to 5, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:08.693831 14204 raft_consensus.cc:1050] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3: attempting to promote NON_VOTER a95f1d3af33f4c879fd290ca917d6329 to VOTER
I20240502 07:59:08.693773 11259 raft_consensus.cc:2932] T f7335ac0209c4e28b6751bb01d34d665 P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:08.694172 14204 consensus_queue.cc:237] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } }
I20240502 07:59:08.695458 11655 raft_consensus.cc:1261] T d37b8a7f016647a7a03baedc37ed5419 P a95f1d3af33f4c879fd290ca917d6329 [term 1 LEARNER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:08.696992 9644 raft_consensus.cc:1261] T d37b8a7f016647a7a03baedc37ed5419 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:08.697173 9514 raft_consensus.cc:1261] T d37b8a7f016647a7a03baedc37ed5419 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:08.697521 13439 consensus_queue.cc:1035] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:08.697819 13439 consensus_queue.cc:1035] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:08.698033 14204 consensus_queue.cc:1035] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:08.699429 9646 consensus_queue.cc:237] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } }
I20240502 07:59:08.700477 9514 raft_consensus.cc:1261] T f7335ac0209c4e28b6751bb01d34d665 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:08.700567 11259 raft_consensus.cc:1261] T f7335ac0209c4e28b6751bb01d34d665 P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:08.700954 14217 consensus_queue.cc:1035] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:08.700876 14204 raft_consensus.cc:2932] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:08.701244 14217 consensus_queue.cc:1035] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:08.702934 14217 raft_consensus.cc:2932] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:08.702960 11655 raft_consensus.cc:2932] T d37b8a7f016647a7a03baedc37ed5419 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:08.703136 9514 raft_consensus.cc:2932] T d37b8a7f016647a7a03baedc37ed5419 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:08.703253 9369 catalog_manager.cc:5484] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 reported cstate change: config changed from index 4 to 5, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1472c64688914149a318c7b8850a3df3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:08.705310 9358 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet f7335ac0209c4e28b6751bb01d34d665 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:08.705299 9644 raft_consensus.cc:2932] T d37b8a7f016647a7a03baedc37ed5419 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:08.705847 9367 catalog_manager.cc:5484] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a reported cstate change: config changed from index 5 to 6, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:08.706921 9777 consensus_queue.cc:237] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } }
I20240502 07:59:08.707227 9514 raft_consensus.cc:2932] T f7335ac0209c4e28b6751bb01d34d665 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:08.708192 9515 raft_consensus.cc:1261] T d37b8a7f016647a7a03baedc37ed5419 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:08.708447 9886 tablet_service.cc:1514] Processing DeleteTablet for tablet f7335ac0209c4e28b6751bb01d34d665 with delete_type TABLET_DATA_TOMBSTONED (TS 6cc5b9962ed24441bcba0c07b9f5438e not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:41940
I20240502 07:59:08.708782 14256 tablet_replica.cc:330] T f7335ac0209c4e28b6751bb01d34d665 P 6cc5b9962ed24441bcba0c07b9f5438e: stopping tablet replica
I20240502 07:59:08.708916 14256 raft_consensus.cc:2229] T f7335ac0209c4e28b6751bb01d34d665 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:08.708878 11259 raft_consensus.cc:2932] T f7335ac0209c4e28b6751bb01d34d665 P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:08.709057 14256 raft_consensus.cc:2258] T f7335ac0209c4e28b6751bb01d34d665 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:08.709421 14207 consensus_queue.cc:1035] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:08.709815 14256 ts_tablet_manager.cc:1875] T f7335ac0209c4e28b6751bb01d34d665 P 6cc5b9962ed24441bcba0c07b9f5438e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:08.710697 11655 raft_consensus.cc:1261] T d37b8a7f016647a7a03baedc37ed5419 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:08.711123 14207 consensus_queue.cc:1035] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:08.711870 14256 ts_tablet_manager.cc:1888] T f7335ac0209c4e28b6751bb01d34d665 P 6cc5b9962ed24441bcba0c07b9f5438e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 07:59:08.711938 14256 log.cc:1196] T f7335ac0209c4e28b6751bb01d34d665 P 6cc5b9962ed24441bcba0c07b9f5438e: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal/wals/f7335ac0209c4e28b6751bb01d34d665
I20240502 07:59:08.712745 9358 catalog_manager.cc:4830] TS 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379): tablet f7335ac0209c4e28b6751bb01d34d665 (table rebalance_test_table_7 [id=7100a319a6ec42499621cff56501ef8f]) successfully deleted
I20240502 07:59:08.713128 14205 raft_consensus.cc:2932] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:08.713570 11655 raft_consensus.cc:2932] T d37b8a7f016647a7a03baedc37ed5419 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:08.714314 9360 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet d37b8a7f016647a7a03baedc37ed5419 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:08.715291 9367 catalog_manager.cc:5484] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 reported cstate change: config changed from index 5 to 6, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New cstate: current_term: 1 leader_uuid: "1472c64688914149a318c7b8850a3df3" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:08.715452 9515 raft_consensus.cc:2932] T d37b8a7f016647a7a03baedc37ed5419 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:08.718011 9624 tablet_service.cc:1514] Processing DeleteTablet for tablet d37b8a7f016647a7a03baedc37ed5419 with delete_type TABLET_DATA_TOMBSTONED (TS 8337af1943564c0dad18b45a597b805a not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:53862
I20240502 07:59:08.718232 14325 tablet_replica.cc:330] T d37b8a7f016647a7a03baedc37ed5419 P 8337af1943564c0dad18b45a597b805a: stopping tablet replica
I20240502 07:59:08.718372 14325 raft_consensus.cc:2229] T d37b8a7f016647a7a03baedc37ed5419 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:08.718498 14325 raft_consensus.cc:2258] T d37b8a7f016647a7a03baedc37ed5419 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:08.719013 14325 ts_tablet_manager.cc:1875] T d37b8a7f016647a7a03baedc37ed5419 P 8337af1943564c0dad18b45a597b805a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:08.720515 14325 ts_tablet_manager.cc:1888] T d37b8a7f016647a7a03baedc37ed5419 P 8337af1943564c0dad18b45a597b805a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 07:59:08.720574 14325 log.cc:1196] T d37b8a7f016647a7a03baedc37ed5419 P 8337af1943564c0dad18b45a597b805a: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal/wals/d37b8a7f016647a7a03baedc37ed5419
I20240502 07:59:08.720965 9358 catalog_manager.cc:4830] TS 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667): tablet d37b8a7f016647a7a03baedc37ed5419 (table rebalance_test_table_3 [id=47c27ad65ee1405898b667c2642e1b3a]) successfully deleted
I20240502 07:59:08.755875 13439 raft_consensus.cc:1050] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3: attempting to promote NON_VOTER 8eb9945f989d4be6b51e1531a1365350 to VOTER
I20240502 07:59:08.756230 13439 consensus_queue.cc:237] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } }
I20240502 07:59:08.757434 11389 raft_consensus.cc:1261] T 70d4db3c3291490db58678522d356688 P 8eb9945f989d4be6b51e1531a1365350 [term 1 LEARNER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:08.757637 9906 raft_consensus.cc:1261] T 70d4db3c3291490db58678522d356688 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:08.758033 14204 consensus_queue.cc:1035] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:08.758069 9644 raft_consensus.cc:1261] T 70d4db3c3291490db58678522d356688 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:08.758366 14204 consensus_queue.cc:1035] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:08.758517 14205 consensus_queue.cc:1035] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:08.760385 14204 raft_consensus.cc:2932] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:08.760705 9644 raft_consensus.cc:2932] T 70d4db3c3291490db58678522d356688 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:08.761277 11389 raft_consensus.cc:2932] T 70d4db3c3291490db58678522d356688 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:08.763186 9906 raft_consensus.cc:2932] T 70d4db3c3291490db58678522d356688 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:08.763741 9365 catalog_manager.cc:5484] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 reported cstate change: config changed from index 4 to 5, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1472c64688914149a318c7b8850a3df3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:08.766389 9777 consensus_queue.cc:237] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } }
I20240502 07:59:08.767462 9906 raft_consensus.cc:1261] T 70d4db3c3291490db58678522d356688 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:08.767478 11389 raft_consensus.cc:1261] T 70d4db3c3291490db58678522d356688 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:08.767792 14205 consensus_queue.cc:1035] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:08.767961 13439 consensus_queue.cc:1035] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:08.769323 14207 raft_consensus.cc:2932] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:08.769763 11389 raft_consensus.cc:2932] T 70d4db3c3291490db58678522d356688 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:08.770615 9360 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 70d4db3c3291490db58678522d356688 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:08.771116 9906 raft_consensus.cc:2932] T 70d4db3c3291490db58678522d356688 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:08.771804 9367 catalog_manager.cc:5484] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 reported cstate change: config changed from index 5 to 6, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New cstate: current_term: 1 leader_uuid: "1472c64688914149a318c7b8850a3df3" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:08.774055 9624 tablet_service.cc:1514] Processing DeleteTablet for tablet 70d4db3c3291490db58678522d356688 with delete_type TABLET_DATA_TOMBSTONED (TS 8337af1943564c0dad18b45a597b805a not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:53862
I20240502 07:59:08.774230 14325 tablet_replica.cc:330] T 70d4db3c3291490db58678522d356688 P 8337af1943564c0dad18b45a597b805a: stopping tablet replica
I20240502 07:59:08.774346 14325 raft_consensus.cc:2229] T 70d4db3c3291490db58678522d356688 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:08.774469 14325 raft_consensus.cc:2258] T 70d4db3c3291490db58678522d356688 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:08.775013 14325 ts_tablet_manager.cc:1875] T 70d4db3c3291490db58678522d356688 P 8337af1943564c0dad18b45a597b805a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:08.776757 14325 ts_tablet_manager.cc:1888] T 70d4db3c3291490db58678522d356688 P 8337af1943564c0dad18b45a597b805a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 07:59:08.776809 14325 log.cc:1196] T 70d4db3c3291490db58678522d356688 P 8337af1943564c0dad18b45a597b805a: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal/wals/70d4db3c3291490db58678522d356688
I20240502 07:59:08.777299 9358 catalog_manager.cc:4830] TS 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667): tablet 70d4db3c3291490db58678522d356688 (table rebalance_test_table_9 [id=f0ea2346b9b9482e8343d199275d5819]) successfully deleted
I20240502 07:59:08.813684 9644 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "427c037fa09c4b94b7bc6c71bdb61c2f"
dest_uuid: "8337af1943564c0dad18b45a597b805a"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:37596
I20240502 07:59:08.813776 9644 raft_consensus.cc:595] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Received request to transfer leadership
I20240502 07:59:08.831959 9644 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "427c037fa09c4b94b7bc6c71bdb61c2f"
dest_uuid: "8337af1943564c0dad18b45a597b805a"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:37608
I20240502 07:59:08.832051 9644 raft_consensus.cc:595] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Received request to transfer leadership
I20240502 07:59:08.870589 11572 raft_consensus.cc:1050] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e: attempting to promote NON_VOTER a95f1d3af33f4c879fd290ca917d6329 to VOTER
I20240502 07:59:08.871019 11572 consensus_queue.cc:237] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } }
I20240502 07:59:08.872416 11655 raft_consensus.cc:1261] T 265eca6fff644d83857ed48a04864a58 P a95f1d3af33f4c879fd290ca917d6329 [term 1 LEARNER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:08.872462 9777 raft_consensus.cc:1261] T 265eca6fff644d83857ed48a04864a58 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:08.872539 9906 raft_consensus.cc:1261] T 265eca6fff644d83857ed48a04864a58 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:08.872844 14319 consensus_queue.cc:1035] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:08.873046 14321 consensus_queue.cc:1035] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:08.873219 14318 consensus_queue.cc:1035] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:08.875774 14319 raft_consensus.cc:2932] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:08.876211 11655 raft_consensus.cc:2932] T 265eca6fff644d83857ed48a04864a58 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:08.876461 9906 raft_consensus.cc:2932] T 265eca6fff644d83857ed48a04864a58 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:08.877645 9777 raft_consensus.cc:2932] T 265eca6fff644d83857ed48a04864a58 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:08.878643 9369 catalog_manager.cc:5484] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e reported cstate change: config changed from index 4 to 5, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:08.881696 9515 consensus_queue.cc:237] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } }
I20240502 07:59:08.882162 14219 raft_consensus.cc:979] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a: : Instructing follower 6cc5b9962ed24441bcba0c07b9f5438e to start an election
I20240502 07:59:08.882249 14219 raft_consensus.cc:1067] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Signalling peer 6cc5b9962ed24441bcba0c07b9f5438e to start an election
I20240502 07:59:08.882910 11655 raft_consensus.cc:1261] T 265eca6fff644d83857ed48a04864a58 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:08.882918 9777 raft_consensus.cc:1261] T 265eca6fff644d83857ed48a04864a58 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:08.883277 11572 consensus_queue.cc:1035] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:08.883453 14321 consensus_queue.cc:1035] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:08.884384 9906 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "427c037fa09c4b94b7bc6c71bdb61c2f"
dest_uuid: "6cc5b9962ed24441bcba0c07b9f5438e"
from {username='slave'} at 127.3.139.66:58991
I20240502 07:59:08.884526 9906 raft_consensus.cc:482] T 427c037fa09c4b94b7bc6c71bdb61c2f P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:08.884589 9906 raft_consensus.cc:3037] T 427c037fa09c4b94b7bc6c71bdb61c2f P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:08.884948 14318 raft_consensus.cc:2932] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:08.885375 9906 raft_consensus.cc:504] T 427c037fa09c4b94b7bc6c71bdb61c2f P 6cc5b9962ed24441bcba0c07b9f5438e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } }
I20240502 07:59:08.885741 9906 leader_election.cc:288] T 427c037fa09c4b94b7bc6c71bdb61c2f P 6cc5b9962ed24441bcba0c07b9f5438e [CANDIDATE]: Term 2 election: Requested vote from peers 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667), 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191), 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70:45329)
I20240502 07:59:08.886112 9644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "427c037fa09c4b94b7bc6c71bdb61c2f" candidate_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "8337af1943564c0dad18b45a597b805a"
I20240502 07:59:08.886222 9644 raft_consensus.cc:3032] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:59:08.886258 9644 raft_consensus.cc:729] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8337af1943564c0dad18b45a597b805a, State: Running, Role: LEADER
I20240502 07:59:08.886313 11389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "427c037fa09c4b94b7bc6c71bdb61c2f" candidate_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "8eb9945f989d4be6b51e1531a1365350"
I20240502 07:59:08.886400 9644 consensus_queue.cc:260] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } }
I20240502 07:59:08.886497 9777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "427c037fa09c4b94b7bc6c71bdb61c2f" candidate_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "1472c64688914149a318c7b8850a3df3"
I20240502 07:59:08.886497 11389 raft_consensus.cc:3037] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:08.886586 9777 raft_consensus.cc:3037] T 427c037fa09c4b94b7bc6c71bdb61c2f P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:08.886595 9644 raft_consensus.cc:3037] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:08.887022 9367 catalog_manager.cc:5484] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e reported cstate change: config changed from index 5 to 6, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:08.887410 9777 raft_consensus.cc:2454] T 427c037fa09c4b94b7bc6c71bdb61c2f P 1472c64688914149a318c7b8850a3df3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cc5b9962ed24441bcba0c07b9f5438e in term 2.
I20240502 07:59:08.887565 9644 raft_consensus.cc:2454] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cc5b9962ed24441bcba0c07b9f5438e in term 2.
I20240502 07:59:08.887717 11389 raft_consensus.cc:2454] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8eb9945f989d4be6b51e1531a1365350 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cc5b9962ed24441bcba0c07b9f5438e in term 2.
I20240502 07:59:08.888000 9842 leader_election.cc:302] T 427c037fa09c4b94b7bc6c71bdb61c2f P 6cc5b9962ed24441bcba0c07b9f5438e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1472c64688914149a318c7b8850a3df3, 6cc5b9962ed24441bcba0c07b9f5438e, 8eb9945f989d4be6b51e1531a1365350; no voters:
I20240502 07:59:08.888096 9360 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 265eca6fff644d83857ed48a04864a58 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:08.888247 9777 raft_consensus.cc:2932] T 265eca6fff644d83857ed48a04864a58 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:08.888347 14091 raft_consensus.cc:2781] T 427c037fa09c4b94b7bc6c71bdb61c2f P 6cc5b9962ed24441bcba0c07b9f5438e [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:08.888522 14091 raft_consensus.cc:686] T 427c037fa09c4b94b7bc6c71bdb61c2f P 6cc5b9962ed24441bcba0c07b9f5438e [term 2 LEADER]: Becoming Leader. State: Replica: 6cc5b9962ed24441bcba0c07b9f5438e, State: Running, Role: LEADER
I20240502 07:59:08.888656 14091 consensus_queue.cc:237] T 427c037fa09c4b94b7bc6c71bdb61c2f P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } }
I20240502 07:59:08.888622 11655 raft_consensus.cc:2932] T 265eca6fff644d83857ed48a04864a58 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:08.889703 9886 tablet_service.cc:1514] Processing DeleteTablet for tablet 265eca6fff644d83857ed48a04864a58 with delete_type TABLET_DATA_TOMBSTONED (TS 6cc5b9962ed24441bcba0c07b9f5438e not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:41940
I20240502 07:59:08.890015 14256 tablet_replica.cc:330] T 265eca6fff644d83857ed48a04864a58 P 6cc5b9962ed24441bcba0c07b9f5438e: stopping tablet replica
I20240502 07:59:08.890136 14256 raft_consensus.cc:2229] T 265eca6fff644d83857ed48a04864a58 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:08.890262 14256 raft_consensus.cc:2258] T 265eca6fff644d83857ed48a04864a58 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:08.891129 14256 ts_tablet_manager.cc:1875] T 265eca6fff644d83857ed48a04864a58 P 6cc5b9962ed24441bcba0c07b9f5438e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:08.893065 14256 ts_tablet_manager.cc:1888] T 265eca6fff644d83857ed48a04864a58 P 6cc5b9962ed24441bcba0c07b9f5438e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 07:59:08.893112 14256 log.cc:1196] T 265eca6fff644d83857ed48a04864a58 P 6cc5b9962ed24441bcba0c07b9f5438e: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal/wals/265eca6fff644d83857ed48a04864a58
I20240502 07:59:08.893482 9358 catalog_manager.cc:4830] TS 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379): tablet 265eca6fff644d83857ed48a04864a58 (table rebalance_test_table_7 [id=7100a319a6ec42499621cff56501ef8f]) successfully deleted
I20240502 07:59:08.893817 9370 catalog_manager.cc:5484] T 427c037fa09c4b94b7bc6c71bdb61c2f P 6cc5b9962ed24441bcba0c07b9f5438e reported cstate change: term changed from 1 to 2, leader changed from 8337af1943564c0dad18b45a597b805a (127.3.139.66) to 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68). New cstate: current_term: 2 leader_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:09.139168 14319 raft_consensus.cc:1050] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e: attempting to promote NON_VOTER a95f1d3af33f4c879fd290ca917d6329 to VOTER
I20240502 07:59:09.139564 14319 consensus_queue.cc:237] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } }
I20240502 07:59:09.140837 11655 raft_consensus.cc:1261] T ee2e79e677f643409e3c8dd49b9517cf P a95f1d3af33f4c879fd290ca917d6329 [term 1 LEARNER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:09.140862 9906 raft_consensus.cc:1261] T ee2e79e677f643409e3c8dd49b9517cf P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:09.140874 9644 raft_consensus.cc:1261] T ee2e79e677f643409e3c8dd49b9517cf P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:09.141214 14319 consensus_queue.cc:1035] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:09.141413 11572 consensus_queue.cc:1035] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:09.141584 14321 consensus_queue.cc:1035] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:09.143544 14321 raft_consensus.cc:2932] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:09.143975 11655 raft_consensus.cc:2932] T ee2e79e677f643409e3c8dd49b9517cf P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:09.144443 9906 raft_consensus.cc:2932] T ee2e79e677f643409e3c8dd49b9517cf P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:09.146077 9644 raft_consensus.cc:2932] T ee2e79e677f643409e3c8dd49b9517cf P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:09.146471 9370 catalog_manager.cc:5484] T ee2e79e677f643409e3c8dd49b9517cf P a95f1d3af33f4c879fd290ca917d6329 reported cstate change: config changed from index 4 to 5, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:09.150625 9515 consensus_queue.cc:237] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } }
I20240502 07:59:09.151585 9644 raft_consensus.cc:1261] T ee2e79e677f643409e3c8dd49b9517cf P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:09.151726 11655 raft_consensus.cc:1261] T ee2e79e677f643409e3c8dd49b9517cf P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:09.151955 11572 consensus_queue.cc:1035] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:09.152206 14319 consensus_queue.cc:1035] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:09.153286 11572 raft_consensus.cc:2932] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:09.153743 9644 raft_consensus.cc:2932] T ee2e79e677f643409e3c8dd49b9517cf P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:09.154037 11655 raft_consensus.cc:2932] T ee2e79e677f643409e3c8dd49b9517cf P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:09.154762 9360 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet ee2e79e677f643409e3c8dd49b9517cf with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:09.156018 9365 catalog_manager.cc:5484] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e reported cstate change: config changed from index 5 to 6, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:09.158501 9886 tablet_service.cc:1514] Processing DeleteTablet for tablet ee2e79e677f643409e3c8dd49b9517cf with delete_type TABLET_DATA_TOMBSTONED (TS 6cc5b9962ed24441bcba0c07b9f5438e not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:41940
I20240502 07:59:09.158735 14256 tablet_replica.cc:330] T ee2e79e677f643409e3c8dd49b9517cf P 6cc5b9962ed24441bcba0c07b9f5438e: stopping tablet replica
I20240502 07:59:09.158859 14256 raft_consensus.cc:2229] T ee2e79e677f643409e3c8dd49b9517cf P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:09.158986 14256 raft_consensus.cc:2258] T ee2e79e677f643409e3c8dd49b9517cf P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:09.159742 14256 ts_tablet_manager.cc:1875] T ee2e79e677f643409e3c8dd49b9517cf P 6cc5b9962ed24441bcba0c07b9f5438e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:09.161357 14256 ts_tablet_manager.cc:1888] T ee2e79e677f643409e3c8dd49b9517cf P 6cc5b9962ed24441bcba0c07b9f5438e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 07:59:09.161418 14256 log.cc:1196] T ee2e79e677f643409e3c8dd49b9517cf P 6cc5b9962ed24441bcba0c07b9f5438e: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal/wals/ee2e79e677f643409e3c8dd49b9517cf
I20240502 07:59:09.161816 9358 catalog_manager.cc:4830] TS 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379): tablet ee2e79e677f643409e3c8dd49b9517cf (table rebalance_test_table_4 [id=a6211a0188464785886f1990ca644211]) successfully deleted
I20240502 07:59:09.276731 11389 raft_consensus.cc:1261] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8eb9945f989d4be6b51e1531a1365350 [term 2 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 07:59:09.277209 14403 consensus_queue.cc:1035] T 427c037fa09c4b94b7bc6c71bdb61c2f P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:09.378634 9644 raft_consensus.cc:1261] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a [term 2 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 07:59:09.379041 14403 consensus_queue.cc:1035] T 427c037fa09c4b94b7bc6c71bdb61c2f P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:09.381786 9777 raft_consensus.cc:1261] T 427c037fa09c4b94b7bc6c71bdb61c2f P 1472c64688914149a318c7b8850a3df3 [term 2 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 07:59:09.383029 14451 consensus_queue.cc:1035] T 427c037fa09c4b94b7bc6c71bdb61c2f P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:09.798970 9906 consensus_queue.cc:237] T 427c037fa09c4b94b7bc6c71bdb61c2f P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } }
I20240502 07:59:09.800125 11389 raft_consensus.cc:1261] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8eb9945f989d4be6b51e1531a1365350 [term 2 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240502 07:59:09.800122 9777 raft_consensus.cc:1261] T 427c037fa09c4b94b7bc6c71bdb61c2f P 1472c64688914149a318c7b8850a3df3 [term 2 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240502 07:59:09.800549 14403 consensus_queue.cc:1035] T 427c037fa09c4b94b7bc6c71bdb61c2f P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:09.800930 14091 consensus_queue.cc:1035] T 427c037fa09c4b94b7bc6c71bdb61c2f P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:09.803150 14091 raft_consensus.cc:2932] T 427c037fa09c4b94b7bc6c71bdb61c2f P 6cc5b9962ed24441bcba0c07b9f5438e [term 2 LEADER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:09.803755 11389 raft_consensus.cc:2932] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8eb9945f989d4be6b51e1531a1365350 [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:09.805351 9358 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 427c037fa09c4b94b7bc6c71bdb61c2f with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:09.805371 9777 raft_consensus.cc:2932] T 427c037fa09c4b94b7bc6c71bdb61c2f P 1472c64688914149a318c7b8850a3df3 [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:09.805958 9369 catalog_manager.cc:5484] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8eb9945f989d4be6b51e1531a1365350 reported cstate change: config changed from index 5 to 7, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New cstate: current_term: 2 leader_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:09.808894 9624 tablet_service.cc:1514] Processing DeleteTablet for tablet 427c037fa09c4b94b7bc6c71bdb61c2f with delete_type TABLET_DATA_TOMBSTONED (TS 8337af1943564c0dad18b45a597b805a not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:53862
I20240502 07:59:09.809199 14469 tablet_replica.cc:330] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a: stopping tablet replica
I20240502 07:59:09.809320 14469 raft_consensus.cc:2229] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:09.809424 14469 raft_consensus.cc:2258] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:09.809944 14469 ts_tablet_manager.cc:1875] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:09.811671 14469 ts_tablet_manager.cc:1888] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.6
I20240502 07:59:09.811734 14469 log.cc:1196] T 427c037fa09c4b94b7bc6c71bdb61c2f P 8337af1943564c0dad18b45a597b805a: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal/wals/427c037fa09c4b94b7bc6c71bdb61c2f
I20240502 07:59:09.812175 9358 catalog_manager.cc:4830] TS 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667): tablet 427c037fa09c4b94b7bc6c71bdb61c2f (table rebalance_test_table_7 [id=7100a319a6ec42499621cff56501ef8f]) successfully deleted
I20240502 07:59:09.964344 9757 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:09.964354 11240 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:09.971411 11502 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:09.972370 9624 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:09.974227 9886 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:09.975845 12569 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:09.986020 11371 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:09.987278 9493 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:10.001268 11635 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:10.036365 9777 consensus_queue.cc:237] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } }
I20240502 07:59:10.037864 9644 raft_consensus.cc:1261] T 96dab19eeaea4648913045482b3e5e9e P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:10.037920 9906 raft_consensus.cc:1261] T 96dab19eeaea4648913045482b3e5e9e P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:10.038286 14529 consensus_queue.cc:1035] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:10.038480 13439 consensus_queue.cc:1035] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:10.040189 14455 raft_consensus.cc:2932] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } } }
I20240502 07:59:10.040599 9644 raft_consensus.cc:2932] T 96dab19eeaea4648913045482b3e5e9e P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } } }
W20240502 07:59:10.042109 9711 consensus_peers.cc:485] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3 -> Peer 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73:35761): Couldn't send request to peer 09ec2b380cbf4200af90b0eee891c010. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 96dab19eeaea4648913045482b3e5e9e. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:10.043184 9367 catalog_manager.cc:5484] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3 reported cstate change: config changed from index -1 to 4, NON_VOTER 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) added. New cstate: current_term: 1 leader_uuid: "1472c64688914149a318c7b8850a3df3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:10.043237 9906 raft_consensus.cc:2932] T 96dab19eeaea4648913045482b3e5e9e P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } } }
I20240502 07:59:10.050599 9515 consensus_queue.cc:237] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } }
I20240502 07:59:10.051759 9644 raft_consensus.cc:1261] T ee2e79e677f643409e3c8dd49b9517cf P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:10.051771 11654 raft_consensus.cc:1261] T ee2e79e677f643409e3c8dd49b9517cf P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240502 07:59:10.052145 9449 consensus_peers.cc:485] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e -> Peer 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70:45329): Couldn't send request to peer 8eb9945f989d4be6b51e1531a1365350. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ee2e79e677f643409e3c8dd49b9517cf. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:10.052186 14522 consensus_queue.cc:1035] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:10.052378 14538 consensus_queue.cc:1035] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:10.053857 14522 raft_consensus.cc:2932] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } } }
I20240502 07:59:10.054294 9644 raft_consensus.cc:2932] T ee2e79e677f643409e3c8dd49b9517cf P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } } }
I20240502 07:59:10.054325 11654 raft_consensus.cc:2932] T ee2e79e677f643409e3c8dd49b9517cf P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } } }
I20240502 07:59:10.056314 9370 catalog_manager.cc:5484] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e reported cstate change: config changed from index 6 to 7, NON_VOTER 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) added. New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:10.063170 9515 consensus_queue.cc:237] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } }
I20240502 07:59:10.064577 11259 raft_consensus.cc:1261] T a12a084dba23425996e9122c083c1444 P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:10.064740 9906 raft_consensus.cc:1261] T a12a084dba23425996e9122c083c1444 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240502 07:59:10.064810 9450 consensus_peers.cc:485] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e -> Peer 98bef2e94ade4763a66c1ad68259744a (127.3.139.71:46577): Couldn't send request to peer 98bef2e94ade4763a66c1ad68259744a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a12a084dba23425996e9122c083c1444. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:10.064915 14318 consensus_queue.cc:1035] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:10.065232 14318 consensus_queue.cc:1035] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:10.066967 14522 raft_consensus.cc:2932] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } } }
I20240502 07:59:10.067404 9906 raft_consensus.cc:2932] T a12a084dba23425996e9122c083c1444 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } } }
I20240502 07:59:10.068372 11259 raft_consensus.cc:2932] T a12a084dba23425996e9122c083c1444 P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } } }
I20240502 07:59:10.069428 9367 catalog_manager.cc:5484] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e reported cstate change: config changed from index 6 to 7, NON_VOTER 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) added. New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:10.077615 9644 consensus_queue.cc:237] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } }
I20240502 07:59:10.079135 11654 raft_consensus.cc:1261] T d562a250cee247a48e573b25ecf00429 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:10.079135 9906 raft_consensus.cc:1261] T d562a250cee247a48e573b25ecf00429 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240502 07:59:10.079260 9582 consensus_peers.cc:485] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a -> Peer 51e82953ed2247fa9ff24638eab3508a (127.3.139.69:45345): Couldn't send request to peer 51e82953ed2247fa9ff24638eab3508a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d562a250cee247a48e573b25ecf00429. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:10.079520 14555 consensus_queue.cc:1035] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:10.079830 14555 consensus_queue.cc:1035] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:10.081146 14556 raft_consensus.cc:2932] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } } }
I20240502 07:59:10.081612 9906 raft_consensus.cc:2932] T d562a250cee247a48e573b25ecf00429 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } } }
I20240502 07:59:10.083774 9370 catalog_manager.cc:5484] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a reported cstate change: config changed from index 6 to 7, NON_VOTER 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) added. New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:10.087955 11654 raft_consensus.cc:2932] T d562a250cee247a48e573b25ecf00429 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } } }
I20240502 07:59:10.093446 9646 consensus_queue.cc:237] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } }
I20240502 07:59:10.094749 9515 raft_consensus.cc:1261] T 12e3bfe638d249c0934feda41b8a8876 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:10.094782 11389 raft_consensus.cc:1261] T 12e3bfe638d249c0934feda41b8a8876 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240502 07:59:10.095026 9581 consensus_peers.cc:485] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a -> Peer a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72:36271): Couldn't send request to peer a95f1d3af33f4c879fd290ca917d6329. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 12e3bfe638d249c0934feda41b8a8876. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:10.095153 14217 consensus_queue.cc:1035] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:10.095443 14556 consensus_queue.cc:1035] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:10.096947 14217 raft_consensus.cc:2932] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } } }
I20240502 07:59:10.097393 9515 raft_consensus.cc:2932] T 12e3bfe638d249c0934feda41b8a8876 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } } }
I20240502 07:59:10.097783 11389 raft_consensus.cc:2932] T 12e3bfe638d249c0934feda41b8a8876 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } } }
I20240502 07:59:10.099678 9367 catalog_manager.cc:5484] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a reported cstate change: config changed from index 6 to 7, NON_VOTER a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) added. New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:10.106922 9906 consensus_queue.cc:237] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } }
I20240502 07:59:10.108294 9777 raft_consensus.cc:1261] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:10.108335 11259 raft_consensus.cc:1261] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:10.108680 14451 consensus_queue.cc:1035] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:10.108886 14091 consensus_queue.cc:1035] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:10.110675 14403 raft_consensus.cc:2932] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } } }
I20240502 07:59:10.111506 11259 raft_consensus.cc:2932] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } } }
I20240502 07:59:10.111516 9777 raft_consensus.cc:2932] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } } }
I20240502 07:59:10.113391 9370 catalog_manager.cc:5484] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e reported cstate change: config changed from index 6 to 7, NON_VOTER 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) added. New cstate: current_term: 1 leader_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20240502 07:59:10.114975 9842 consensus_peers.cc:485] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e -> Peer 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73:35761): Couldn't send request to peer 09ec2b380cbf4200af90b0eee891c010. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7da9d3b7fe8c40f6a68bb1b7a926d994. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:10.120652 9777 consensus_queue.cc:237] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } }
I20240502 07:59:10.122025 9515 raft_consensus.cc:1261] T d37b8a7f016647a7a03baedc37ed5419 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:10.122040 11654 raft_consensus.cc:1261] T d37b8a7f016647a7a03baedc37ed5419 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240502 07:59:10.122241 9711 consensus_peers.cc:485] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 -> Peer 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70:45329): Couldn't send request to peer 8eb9945f989d4be6b51e1531a1365350. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d37b8a7f016647a7a03baedc37ed5419. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:10.122417 14455 consensus_queue.cc:1035] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:10.122584 14529 consensus_queue.cc:1035] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:10.124277 14455 raft_consensus.cc:2932] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } } }
I20240502 07:59:10.124763 11654 raft_consensus.cc:2932] T d37b8a7f016647a7a03baedc37ed5419 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } } }
I20240502 07:59:10.125114 9515 raft_consensus.cc:2932] T d37b8a7f016647a7a03baedc37ed5419 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } } }
I20240502 07:59:10.126821 9372 catalog_manager.cc:5484] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 reported cstate change: config changed from index 6 to 7, NON_VOTER 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) added. New cstate: current_term: 1 leader_uuid: "1472c64688914149a318c7b8850a3df3" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:10.134435 9777 consensus_queue.cc:237] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } }
I20240502 07:59:10.135668 11520 raft_consensus.cc:1261] T 0e4efe560ef74763b80b70665c11d7f1 P 98bef2e94ade4763a66c1ad68259744a [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240502 07:59:10.135787 9711 consensus_peers.cc:485] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 -> Peer 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73:35761): Couldn't send request to peer 09ec2b380cbf4200af90b0eee891c010. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e4efe560ef74763b80b70665c11d7f1. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:10.136008 9906 raft_consensus.cc:1261] T 0e4efe560ef74763b80b70665c11d7f1 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:10.136032 13439 consensus_queue.cc:1035] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:10.136330 14455 consensus_queue.cc:1035] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:10.137737 14455 raft_consensus.cc:2932] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } } }
I20240502 07:59:10.138108 11520 raft_consensus.cc:2932] T 0e4efe560ef74763b80b70665c11d7f1 P 98bef2e94ade4763a66c1ad68259744a [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } } }
I20240502 07:59:10.139657 9906 raft_consensus.cc:2932] T 0e4efe560ef74763b80b70665c11d7f1 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } } }
I20240502 07:59:10.140565 9367 catalog_manager.cc:5484] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 reported cstate change: config changed from index 6 to 7, NON_VOTER 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) added. New cstate: current_term: 1 leader_uuid: "1472c64688914149a318c7b8850a3df3" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:10.147439 9646 consensus_queue.cc:237] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } }
I20240502 07:59:10.148767 9777 raft_consensus.cc:1261] T 8ceb129e9cbe4c469e7dfba419719556 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240502 07:59:10.148849 9581 consensus_peers.cc:485] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a -> Peer 98bef2e94ade4763a66c1ad68259744a (127.3.139.71:46577): Couldn't send request to peer 98bef2e94ade4763a66c1ad68259744a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8ceb129e9cbe4c469e7dfba419719556. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:10.148936 11259 raft_consensus.cc:1261] T 8ceb129e9cbe4c469e7dfba419719556 P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:10.149283 14555 consensus_queue.cc:1035] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:10.149458 14556 consensus_queue.cc:1035] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:10.151124 14217 raft_consensus.cc:2932] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } } }
I20240502 07:59:10.151546 9777 raft_consensus.cc:2932] T 8ceb129e9cbe4c469e7dfba419719556 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } } }
I20240502 07:59:10.153527 11259 raft_consensus.cc:2932] T 8ceb129e9cbe4c469e7dfba419719556 P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } } }
I20240502 07:59:10.155077 9370 catalog_manager.cc:5484] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a reported cstate change: config changed from index 6 to 7, NON_VOTER 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) added. New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:10.161811 9906 consensus_queue.cc:237] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } }
I20240502 07:59:10.163024 11654 raft_consensus.cc:1261] T 3255d89d51a94426a78364230c9b249b P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240502 07:59:10.163179 9844 consensus_peers.cc:485] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e -> Peer 51e82953ed2247fa9ff24638eab3508a (127.3.139.69:45345): Couldn't send request to peer 51e82953ed2247fa9ff24638eab3508a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3255d89d51a94426a78364230c9b249b. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:10.163252 9646 raft_consensus.cc:1261] T 3255d89d51a94426a78364230c9b249b P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:10.163576 14451 consensus_queue.cc:1035] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:10.163769 14403 consensus_queue.cc:1035] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:10.165419 14451 raft_consensus.cc:2932] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } } }
I20240502 07:59:10.165829 11654 raft_consensus.cc:2932] T 3255d89d51a94426a78364230c9b249b P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } } }
I20240502 07:59:10.167877 9370 catalog_manager.cc:5484] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e reported cstate change: config changed from index 6 to 7, NON_VOTER 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) added. New cstate: current_term: 1 leader_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:10.168618 9646 raft_consensus.cc:2932] T 3255d89d51a94426a78364230c9b249b P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } } }
I20240502 07:59:10.175352 9646 consensus_queue.cc:237] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } }
I20240502 07:59:10.176836 11520 raft_consensus.cc:1261] T 364d227472894712a2ae15f4589a5a5b P 98bef2e94ade4763a66c1ad68259744a [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:10.177305 9515 raft_consensus.cc:1261] T 364d227472894712a2ae15f4589a5a5b P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:10.177327 14555 consensus_queue.cc:1035] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:10.177620 14555 consensus_queue.cc:1035] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:10.179419 14217 raft_consensus.cc:2932] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } } }
I20240502 07:59:10.180478 11520 raft_consensus.cc:2932] T 364d227472894712a2ae15f4589a5a5b P 98bef2e94ade4763a66c1ad68259744a [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } } }
I20240502 07:59:10.181717 9515 raft_consensus.cc:2932] T 364d227472894712a2ae15f4589a5a5b P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } } }
I20240502 07:59:10.181767 9370 catalog_manager.cc:5484] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a reported cstate change: config changed from index 6 to 7, NON_VOTER 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) added. New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20240502 07:59:10.182354 9580 consensus_peers.cc:485] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a -> Peer 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73:35761): Couldn't send request to peer 09ec2b380cbf4200af90b0eee891c010. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 364d227472894712a2ae15f4589a5a5b. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:10.189186 9646 consensus_queue.cc:237] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } }
I20240502 07:59:10.190537 11389 raft_consensus.cc:1261] T 50f059668c6b4205a15d2c076e2789b0 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:10.190536 9906 raft_consensus.cc:1261] T 50f059668c6b4205a15d2c076e2789b0 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:10.190903 14555 consensus_queue.cc:1035] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:10.191112 14217 consensus_queue.cc:1035] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20240502 07:59:10.191337 9580 consensus_peers.cc:485] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a -> Peer 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73:35761): Couldn't send request to peer 09ec2b380cbf4200af90b0eee891c010. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 50f059668c6b4205a15d2c076e2789b0. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:10.192783 14555 raft_consensus.cc:2932] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } } }
I20240502 07:59:10.193171 11389 raft_consensus.cc:2932] T 50f059668c6b4205a15d2c076e2789b0 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } } }
I20240502 07:59:10.193285 9906 raft_consensus.cc:2932] T 50f059668c6b4205a15d2c076e2789b0 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } } }
I20240502 07:59:10.195564 9370 catalog_manager.cc:5484] T 50f059668c6b4205a15d2c076e2789b0 P 8eb9945f989d4be6b51e1531a1365350 reported cstate change: config changed from index 6 to 7, NON_VOTER 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) added. New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } } }
I20240502 07:59:10.202714 9777 consensus_queue.cc:237] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } }
I20240502 07:59:10.204087 9906 raft_consensus.cc:1261] T 92f37161960e4ee894c75d9034434001 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:10.204100 11389 raft_consensus.cc:1261] T 92f37161960e4ee894c75d9034434001 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240502 07:59:10.204180 9712 consensus_peers.cc:485] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 -> Peer 98bef2e94ade4763a66c1ad68259744a (127.3.139.71:46577): Couldn't send request to peer 98bef2e94ade4763a66c1ad68259744a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 92f37161960e4ee894c75d9034434001. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:10.204468 14529 consensus_queue.cc:1035] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:10.204640 14455 consensus_queue.cc:1035] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:10.206260 13439 raft_consensus.cc:2932] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } } }
I20240502 07:59:10.206646 9906 raft_consensus.cc:2932] T 92f37161960e4ee894c75d9034434001 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } } }
I20240502 07:59:10.206686 11389 raft_consensus.cc:2932] T 92f37161960e4ee894c75d9034434001 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } } }
I20240502 07:59:10.208508 9367 catalog_manager.cc:5484] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 reported cstate change: config changed from index 6 to 7, NON_VOTER 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) added. New cstate: current_term: 1 leader_uuid: "1472c64688914149a318c7b8850a3df3" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:10.215728 9515 consensus_queue.cc:237] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } }
I20240502 07:59:10.216956 9646 raft_consensus.cc:1261] T bc896a65284c48cd864bb6b57dbe5ca0 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:10.217005 11654 raft_consensus.cc:1261] T bc896a65284c48cd864bb6b57dbe5ca0 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:10.217301 14538 consensus_queue.cc:1035] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20240502 07:59:10.217463 9451 consensus_peers.cc:485] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e -> Peer 51e82953ed2247fa9ff24638eab3508a (127.3.139.69:45345): Couldn't send request to peer 51e82953ed2247fa9ff24638eab3508a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bc896a65284c48cd864bb6b57dbe5ca0. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:10.217746 14538 consensus_queue.cc:1035] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:10.219137 14522 raft_consensus.cc:2932] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } } }
I20240502 07:59:10.219538 9646 raft_consensus.cc:2932] T bc896a65284c48cd864bb6b57dbe5ca0 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } } }
I20240502 07:59:10.221458 11654 raft_consensus.cc:2932] T bc896a65284c48cd864bb6b57dbe5ca0 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } } }
I20240502 07:59:10.221717 9370 catalog_manager.cc:5484] T bc896a65284c48cd864bb6b57dbe5ca0 P 8337af1943564c0dad18b45a597b805a reported cstate change: config changed from index 6 to 7, NON_VOTER 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) added. New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } } }
I20240502 07:59:10.229529 9515 consensus_queue.cc:237] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } }
I20240502 07:59:10.230706 11654 raft_consensus.cc:1261] T 265eca6fff644d83857ed48a04864a58 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:10.230713 9777 raft_consensus.cc:1261] T 265eca6fff644d83857ed48a04864a58 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240502 07:59:10.230931 9450 consensus_peers.cc:485] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e -> Peer 98bef2e94ade4763a66c1ad68259744a (127.3.139.71:46577): Couldn't send request to peer 98bef2e94ade4763a66c1ad68259744a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 265eca6fff644d83857ed48a04864a58. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:10.231073 14318 consensus_queue.cc:1035] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:10.231379 14318 consensus_queue.cc:1035] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:10.232733 14522 raft_consensus.cc:2932] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } } }
I20240502 07:59:10.233112 9777 raft_consensus.cc:2932] T 265eca6fff644d83857ed48a04864a58 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } } }
I20240502 07:59:10.233263 11654 raft_consensus.cc:2932] T 265eca6fff644d83857ed48a04864a58 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } } }
I20240502 07:59:10.235574 9367 catalog_manager.cc:5484] T 265eca6fff644d83857ed48a04864a58 P 1472c64688914149a318c7b8850a3df3 reported cstate change: config changed from index 6 to 7, NON_VOTER 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) added. New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } } }
I20240502 07:59:10.523772 14735 ts_tablet_manager.cc:902] T a12a084dba23425996e9122c083c1444 P 98bef2e94ade4763a66c1ad68259744a: Initiating tablet copy from peer a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551)
I20240502 07:59:10.524631 14735 tablet_copy_client.cc:308] T a12a084dba23425996e9122c083c1444 P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.65:39551
I20240502 07:59:10.525074 9534 tablet_copy_service.cc:140] P a201db721bda4dc3a4422dc8fa9bf72e: Received BeginTabletCopySession request for tablet a12a084dba23425996e9122c083c1444 from peer 98bef2e94ade4763a66c1ad68259744a ({username='slave'} at 127.3.139.71:43779)
I20240502 07:59:10.525188 9534 tablet_copy_service.cc:161] P a201db721bda4dc3a4422dc8fa9bf72e: Beginning new tablet copy session on tablet a12a084dba23425996e9122c083c1444 from peer 98bef2e94ade4763a66c1ad68259744a at {username='slave'} at 127.3.139.71:43779: session id = 98bef2e94ade4763a66c1ad68259744a-a12a084dba23425996e9122c083c1444
I20240502 07:59:10.526144 9534 tablet_copy_source_session.cc:205] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:10.526969 14735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a12a084dba23425996e9122c083c1444. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:10.529439 14735 tablet_copy_client.cc:783] T a12a084dba23425996e9122c083c1444 P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:10.529632 14735 tablet_copy_client.cc:647] T a12a084dba23425996e9122c083c1444 P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:10.530681 14744 tablet_copy_client.cc:1002] T a12a084dba23425996e9122c083c1444 P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Injecting latency into file download: 1500
I20240502 07:59:10.588670 14804 ts_tablet_manager.cc:902] T 96dab19eeaea4648913045482b3e5e9e P 09ec2b380cbf4200af90b0eee891c010: Initiating tablet copy from peer 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191)
I20240502 07:59:10.589375 14804 tablet_copy_client.cc:308] T 96dab19eeaea4648913045482b3e5e9e P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.67:43191
I20240502 07:59:10.593391 9797 tablet_copy_service.cc:140] P 1472c64688914149a318c7b8850a3df3: Received BeginTabletCopySession request for tablet 96dab19eeaea4648913045482b3e5e9e from peer 09ec2b380cbf4200af90b0eee891c010 ({username='slave'} at 127.3.139.73:38883)
I20240502 07:59:10.593488 9797 tablet_copy_service.cc:161] P 1472c64688914149a318c7b8850a3df3: Beginning new tablet copy session on tablet 96dab19eeaea4648913045482b3e5e9e from peer 09ec2b380cbf4200af90b0eee891c010 at {username='slave'} at 127.3.139.73:38883: session id = 09ec2b380cbf4200af90b0eee891c010-96dab19eeaea4648913045482b3e5e9e
I20240502 07:59:10.594341 9797 tablet_copy_source_session.cc:205] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:10.597769 14804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96dab19eeaea4648913045482b3e5e9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:10.603333 14804 tablet_copy_client.cc:783] T 96dab19eeaea4648913045482b3e5e9e P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:10.603724 14804 tablet_copy_client.cc:647] T 96dab19eeaea4648913045482b3e5e9e P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:10.604924 14806 tablet_copy_client.cc:1002] T 96dab19eeaea4648913045482b3e5e9e P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Injecting latency into file download: 1500
I20240502 07:59:10.621724 14812 ts_tablet_manager.cc:902] T ee2e79e677f643409e3c8dd49b9517cf P 8eb9945f989d4be6b51e1531a1365350: Initiating tablet copy from peer a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551)
I20240502 07:59:10.622128 14812 tablet_copy_client.cc:308] T ee2e79e677f643409e3c8dd49b9517cf P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.65:39551
I20240502 07:59:10.622601 9534 tablet_copy_service.cc:140] P a201db721bda4dc3a4422dc8fa9bf72e: Received BeginTabletCopySession request for tablet ee2e79e677f643409e3c8dd49b9517cf from peer 8eb9945f989d4be6b51e1531a1365350 ({username='slave'} at 127.3.139.70:59727)
I20240502 07:59:10.622681 9534 tablet_copy_service.cc:161] P a201db721bda4dc3a4422dc8fa9bf72e: Beginning new tablet copy session on tablet ee2e79e677f643409e3c8dd49b9517cf from peer 8eb9945f989d4be6b51e1531a1365350 at {username='slave'} at 127.3.139.70:59727: session id = 8eb9945f989d4be6b51e1531a1365350-ee2e79e677f643409e3c8dd49b9517cf
I20240502 07:59:10.623412 9534 tablet_copy_source_session.cc:205] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:10.624189 14812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee2e79e677f643409e3c8dd49b9517cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:10.626066 14812 tablet_copy_client.cc:783] T ee2e79e677f643409e3c8dd49b9517cf P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:10.626196 14812 tablet_copy_client.cc:647] T ee2e79e677f643409e3c8dd49b9517cf P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:10.627041 14816 tablet_copy_client.cc:1002] T ee2e79e677f643409e3c8dd49b9517cf P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Injecting latency into file download: 1500
I20240502 07:59:10.635622 14817 ts_tablet_manager.cc:902] T bc896a65284c48cd864bb6b57dbe5ca0 P 51e82953ed2247fa9ff24638eab3508a: Initiating tablet copy from peer a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551)
I20240502 07:59:10.636027 14817 tablet_copy_client.cc:308] T bc896a65284c48cd864bb6b57dbe5ca0 P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.65:39551
I20240502 07:59:10.636416 9534 tablet_copy_service.cc:140] P a201db721bda4dc3a4422dc8fa9bf72e: Received BeginTabletCopySession request for tablet bc896a65284c48cd864bb6b57dbe5ca0 from peer 51e82953ed2247fa9ff24638eab3508a ({username='slave'} at 127.3.139.69:55681)
I20240502 07:59:10.636502 9534 tablet_copy_service.cc:161] P a201db721bda4dc3a4422dc8fa9bf72e: Beginning new tablet copy session on tablet bc896a65284c48cd864bb6b57dbe5ca0 from peer 51e82953ed2247fa9ff24638eab3508a at {username='slave'} at 127.3.139.69:55681: session id = 51e82953ed2247fa9ff24638eab3508a-bc896a65284c48cd864bb6b57dbe5ca0
I20240502 07:59:10.637161 9534 tablet_copy_source_session.cc:205] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:10.637841 14817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc896a65284c48cd864bb6b57dbe5ca0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:10.639673 14817 tablet_copy_client.cc:783] T bc896a65284c48cd864bb6b57dbe5ca0 P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:10.639815 14817 tablet_copy_client.cc:647] T bc896a65284c48cd864bb6b57dbe5ca0 P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:10.640697 14821 tablet_copy_client.cc:1002] T bc896a65284c48cd864bb6b57dbe5ca0 P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Injecting latency into file download: 1500
I20240502 07:59:10.643795 14822 ts_tablet_manager.cc:902] T 12e3bfe638d249c0934feda41b8a8876 P a95f1d3af33f4c879fd290ca917d6329: Initiating tablet copy from peer 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667)
I20240502 07:59:10.644240 14822 tablet_copy_client.cc:308] T 12e3bfe638d249c0934feda41b8a8876 P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.66:46667
I20240502 07:59:10.644662 9666 tablet_copy_service.cc:140] P 8337af1943564c0dad18b45a597b805a: Received BeginTabletCopySession request for tablet 12e3bfe638d249c0934feda41b8a8876 from peer a95f1d3af33f4c879fd290ca917d6329 ({username='slave'} at 127.3.139.72:42289)
I20240502 07:59:10.644748 9666 tablet_copy_service.cc:161] P 8337af1943564c0dad18b45a597b805a: Beginning new tablet copy session on tablet 12e3bfe638d249c0934feda41b8a8876 from peer a95f1d3af33f4c879fd290ca917d6329 at {username='slave'} at 127.3.139.72:42289: session id = a95f1d3af33f4c879fd290ca917d6329-12e3bfe638d249c0934feda41b8a8876
I20240502 07:59:10.645388 9666 tablet_copy_source_session.cc:205] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:10.646236 14822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12e3bfe638d249c0934feda41b8a8876. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:10.647920 14822 tablet_copy_client.cc:783] T 12e3bfe638d249c0934feda41b8a8876 P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:10.648058 14822 tablet_copy_client.cc:647] T 12e3bfe638d249c0934feda41b8a8876 P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:10.648816 14827 ts_tablet_manager.cc:902] T d562a250cee247a48e573b25ecf00429 P 51e82953ed2247fa9ff24638eab3508a: Initiating tablet copy from peer 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667)
I20240502 07:59:10.649084 14826 tablet_copy_client.cc:1002] T 12e3bfe638d249c0934feda41b8a8876 P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Injecting latency into file download: 1500
I20240502 07:59:10.649262 14827 tablet_copy_client.cc:308] T d562a250cee247a48e573b25ecf00429 P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.66:46667
I20240502 07:59:10.649658 9666 tablet_copy_service.cc:140] P 8337af1943564c0dad18b45a597b805a: Received BeginTabletCopySession request for tablet d562a250cee247a48e573b25ecf00429 from peer 51e82953ed2247fa9ff24638eab3508a ({username='slave'} at 127.3.139.69:35563)
I20240502 07:59:10.649740 9666 tablet_copy_service.cc:161] P 8337af1943564c0dad18b45a597b805a: Beginning new tablet copy session on tablet d562a250cee247a48e573b25ecf00429 from peer 51e82953ed2247fa9ff24638eab3508a at {username='slave'} at 127.3.139.69:35563: session id = 51e82953ed2247fa9ff24638eab3508a-d562a250cee247a48e573b25ecf00429
I20240502 07:59:10.650283 9666 tablet_copy_source_session.cc:205] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:10.650909 14827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d562a250cee247a48e573b25ecf00429. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:10.652366 14827 tablet_copy_client.cc:783] T d562a250cee247a48e573b25ecf00429 P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:10.652500 14827 tablet_copy_client.cc:647] T d562a250cee247a48e573b25ecf00429 P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:10.653352 14831 tablet_copy_client.cc:1002] T d562a250cee247a48e573b25ecf00429 P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Injecting latency into file download: 1500
I20240502 07:59:10.659092 14832 ts_tablet_manager.cc:902] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 09ec2b380cbf4200af90b0eee891c010: Initiating tablet copy from peer 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379)
I20240502 07:59:10.659528 14832 tablet_copy_client.cc:308] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.68:37379
I20240502 07:59:10.662568 9928 tablet_copy_service.cc:140] P 6cc5b9962ed24441bcba0c07b9f5438e: Received BeginTabletCopySession request for tablet 7da9d3b7fe8c40f6a68bb1b7a926d994 from peer 09ec2b380cbf4200af90b0eee891c010 ({username='slave'} at 127.3.139.73:40787)
I20240502 07:59:10.662678 9928 tablet_copy_service.cc:161] P 6cc5b9962ed24441bcba0c07b9f5438e: Beginning new tablet copy session on tablet 7da9d3b7fe8c40f6a68bb1b7a926d994 from peer 09ec2b380cbf4200af90b0eee891c010 at {username='slave'} at 127.3.139.73:40787: session id = 09ec2b380cbf4200af90b0eee891c010-7da9d3b7fe8c40f6a68bb1b7a926d994
I20240502 07:59:10.663360 9928 tablet_copy_source_session.cc:205] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:10.664032 14832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7da9d3b7fe8c40f6a68bb1b7a926d994. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:10.665539 14832 tablet_copy_client.cc:783] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:10.665654 14832 tablet_copy_client.cc:647] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:10.666627 14836 tablet_copy_client.cc:1002] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Injecting latency into file download: 1500
I20240502 07:59:10.697044 14837 ts_tablet_manager.cc:902] T 265eca6fff644d83857ed48a04864a58 P 98bef2e94ade4763a66c1ad68259744a: Initiating tablet copy from peer a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551)
I20240502 07:59:10.697571 14837 tablet_copy_client.cc:308] T 265eca6fff644d83857ed48a04864a58 P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.65:39551
I20240502 07:59:10.698014 9534 tablet_copy_service.cc:140] P a201db721bda4dc3a4422dc8fa9bf72e: Received BeginTabletCopySession request for tablet 265eca6fff644d83857ed48a04864a58 from peer 98bef2e94ade4763a66c1ad68259744a ({username='slave'} at 127.3.139.71:43779)
I20240502 07:59:10.698122 9534 tablet_copy_service.cc:161] P a201db721bda4dc3a4422dc8fa9bf72e: Beginning new tablet copy session on tablet 265eca6fff644d83857ed48a04864a58 from peer 98bef2e94ade4763a66c1ad68259744a at {username='slave'} at 127.3.139.71:43779: session id = 98bef2e94ade4763a66c1ad68259744a-265eca6fff644d83857ed48a04864a58
I20240502 07:59:10.698756 14842 ts_tablet_manager.cc:902] T 364d227472894712a2ae15f4589a5a5b P 09ec2b380cbf4200af90b0eee891c010: Initiating tablet copy from peer 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667)
I20240502 07:59:10.699040 9534 tablet_copy_source_session.cc:205] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:10.699208 14842 tablet_copy_client.cc:308] T 364d227472894712a2ae15f4589a5a5b P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.66:46667
I20240502 07:59:10.699855 14837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 265eca6fff644d83857ed48a04864a58. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:10.701484 14837 tablet_copy_client.cc:783] T 265eca6fff644d83857ed48a04864a58 P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:10.701617 14837 tablet_copy_client.cc:647] T 265eca6fff644d83857ed48a04864a58 P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:10.702431 9666 tablet_copy_service.cc:140] P 8337af1943564c0dad18b45a597b805a: Received BeginTabletCopySession request for tablet 364d227472894712a2ae15f4589a5a5b from peer 09ec2b380cbf4200af90b0eee891c010 ({username='slave'} at 127.3.139.73:58997)
I20240502 07:59:10.702525 9666 tablet_copy_service.cc:161] P 8337af1943564c0dad18b45a597b805a: Beginning new tablet copy session on tablet 364d227472894712a2ae15f4589a5a5b from peer 09ec2b380cbf4200af90b0eee891c010 at {username='slave'} at 127.3.139.73:58997: session id = 09ec2b380cbf4200af90b0eee891c010-364d227472894712a2ae15f4589a5a5b
I20240502 07:59:10.702657 14841 tablet_copy_client.cc:1002] T 265eca6fff644d83857ed48a04864a58 P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Injecting latency into file download: 1500
I20240502 07:59:10.703197 9666 tablet_copy_source_session.cc:205] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:10.703848 14842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 364d227472894712a2ae15f4589a5a5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:10.705546 14842 tablet_copy_client.cc:783] T 364d227472894712a2ae15f4589a5a5b P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:10.705720 14842 tablet_copy_client.cc:647] T 364d227472894712a2ae15f4589a5a5b P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:10.706146 14847 ts_tablet_manager.cc:902] T d37b8a7f016647a7a03baedc37ed5419 P 8eb9945f989d4be6b51e1531a1365350: Initiating tablet copy from peer 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191)
I20240502 07:59:10.706617 14847 tablet_copy_client.cc:308] T d37b8a7f016647a7a03baedc37ed5419 P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.67:43191
I20240502 07:59:10.706650 14846 tablet_copy_client.cc:1002] T 364d227472894712a2ae15f4589a5a5b P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Injecting latency into file download: 1500
I20240502 07:59:10.707041 9797 tablet_copy_service.cc:140] P 1472c64688914149a318c7b8850a3df3: Received BeginTabletCopySession request for tablet d37b8a7f016647a7a03baedc37ed5419 from peer 8eb9945f989d4be6b51e1531a1365350 ({username='slave'} at 127.3.139.70:41973)
I20240502 07:59:10.707162 9797 tablet_copy_service.cc:161] P 1472c64688914149a318c7b8850a3df3: Beginning new tablet copy session on tablet d37b8a7f016647a7a03baedc37ed5419 from peer 8eb9945f989d4be6b51e1531a1365350 at {username='slave'} at 127.3.139.70:41973: session id = 8eb9945f989d4be6b51e1531a1365350-d37b8a7f016647a7a03baedc37ed5419
I20240502 07:59:10.707814 9797 tablet_copy_source_session.cc:205] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:10.708498 14847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d37b8a7f016647a7a03baedc37ed5419. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:10.709957 14847 tablet_copy_client.cc:783] T d37b8a7f016647a7a03baedc37ed5419 P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:10.710072 14847 tablet_copy_client.cc:647] T d37b8a7f016647a7a03baedc37ed5419 P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:10.710958 14851 tablet_copy_client.cc:1002] T d37b8a7f016647a7a03baedc37ed5419 P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Injecting latency into file download: 1500
I20240502 07:59:10.712610 14852 ts_tablet_manager.cc:902] T 50f059668c6b4205a15d2c076e2789b0 P 09ec2b380cbf4200af90b0eee891c010: Initiating tablet copy from peer 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667)
I20240502 07:59:10.713011 14852 tablet_copy_client.cc:308] T 50f059668c6b4205a15d2c076e2789b0 P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.66:46667
I20240502 07:59:10.713407 9666 tablet_copy_service.cc:140] P 8337af1943564c0dad18b45a597b805a: Received BeginTabletCopySession request for tablet 50f059668c6b4205a15d2c076e2789b0 from peer 09ec2b380cbf4200af90b0eee891c010 ({username='slave'} at 127.3.139.73:58997)
I20240502 07:59:10.713481 9666 tablet_copy_service.cc:161] P 8337af1943564c0dad18b45a597b805a: Beginning new tablet copy session on tablet 50f059668c6b4205a15d2c076e2789b0 from peer 09ec2b380cbf4200af90b0eee891c010 at {username='slave'} at 127.3.139.73:58997: session id = 09ec2b380cbf4200af90b0eee891c010-50f059668c6b4205a15d2c076e2789b0
I20240502 07:59:10.714069 9666 tablet_copy_source_session.cc:205] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:10.714658 14852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50f059668c6b4205a15d2c076e2789b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:10.716161 14852 tablet_copy_client.cc:783] T 50f059668c6b4205a15d2c076e2789b0 P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:10.716280 14852 tablet_copy_client.cc:647] T 50f059668c6b4205a15d2c076e2789b0 P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:10.717146 14856 tablet_copy_client.cc:1002] T 50f059668c6b4205a15d2c076e2789b0 P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Injecting latency into file download: 1500
I20240502 07:59:10.724144 14857 ts_tablet_manager.cc:902] T 0e4efe560ef74763b80b70665c11d7f1 P 09ec2b380cbf4200af90b0eee891c010: Initiating tablet copy from peer 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191)
I20240502 07:59:10.724545 14857 tablet_copy_client.cc:308] T 0e4efe560ef74763b80b70665c11d7f1 P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.67:43191
I20240502 07:59:10.724939 9797 tablet_copy_service.cc:140] P 1472c64688914149a318c7b8850a3df3: Received BeginTabletCopySession request for tablet 0e4efe560ef74763b80b70665c11d7f1 from peer 09ec2b380cbf4200af90b0eee891c010 ({username='slave'} at 127.3.139.73:38883)
I20240502 07:59:10.725018 9797 tablet_copy_service.cc:161] P 1472c64688914149a318c7b8850a3df3: Beginning new tablet copy session on tablet 0e4efe560ef74763b80b70665c11d7f1 from peer 09ec2b380cbf4200af90b0eee891c010 at {username='slave'} at 127.3.139.73:38883: session id = 09ec2b380cbf4200af90b0eee891c010-0e4efe560ef74763b80b70665c11d7f1
I20240502 07:59:10.725625 9797 tablet_copy_source_session.cc:205] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:10.726226 14857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e4efe560ef74763b80b70665c11d7f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:10.727559 14857 tablet_copy_client.cc:783] T 0e4efe560ef74763b80b70665c11d7f1 P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:10.727677 14857 tablet_copy_client.cc:647] T 0e4efe560ef74763b80b70665c11d7f1 P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:10.728554 14861 tablet_copy_client.cc:1002] T 0e4efe560ef74763b80b70665c11d7f1 P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Injecting latency into file download: 1500
I20240502 07:59:10.733115 14862 ts_tablet_manager.cc:902] T 8ceb129e9cbe4c469e7dfba419719556 P 98bef2e94ade4763a66c1ad68259744a: Initiating tablet copy from peer 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667)
I20240502 07:59:10.733552 14862 tablet_copy_client.cc:308] T 8ceb129e9cbe4c469e7dfba419719556 P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.66:46667
I20240502 07:59:10.733924 9666 tablet_copy_service.cc:140] P 8337af1943564c0dad18b45a597b805a: Received BeginTabletCopySession request for tablet 8ceb129e9cbe4c469e7dfba419719556 from peer 98bef2e94ade4763a66c1ad68259744a ({username='slave'} at 127.3.139.71:52353)
I20240502 07:59:10.734000 9666 tablet_copy_service.cc:161] P 8337af1943564c0dad18b45a597b805a: Beginning new tablet copy session on tablet 8ceb129e9cbe4c469e7dfba419719556 from peer 98bef2e94ade4763a66c1ad68259744a at {username='slave'} at 127.3.139.71:52353: session id = 98bef2e94ade4763a66c1ad68259744a-8ceb129e9cbe4c469e7dfba419719556
I20240502 07:59:10.734601 9666 tablet_copy_source_session.cc:205] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:10.735349 14862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ceb129e9cbe4c469e7dfba419719556. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:10.737015 14862 tablet_copy_client.cc:783] T 8ceb129e9cbe4c469e7dfba419719556 P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:10.737201 14862 tablet_copy_client.cc:647] T 8ceb129e9cbe4c469e7dfba419719556 P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:10.738260 14866 tablet_copy_client.cc:1002] T 8ceb129e9cbe4c469e7dfba419719556 P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Injecting latency into file download: 1500
I20240502 07:59:10.739079 14867 ts_tablet_manager.cc:902] T 3255d89d51a94426a78364230c9b249b P 51e82953ed2247fa9ff24638eab3508a: Initiating tablet copy from peer 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379)
I20240502 07:59:10.739527 14867 tablet_copy_client.cc:308] T 3255d89d51a94426a78364230c9b249b P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.68:37379
I20240502 07:59:10.739948 9928 tablet_copy_service.cc:140] P 6cc5b9962ed24441bcba0c07b9f5438e: Received BeginTabletCopySession request for tablet 3255d89d51a94426a78364230c9b249b from peer 51e82953ed2247fa9ff24638eab3508a ({username='slave'} at 127.3.139.69:48381)
I20240502 07:59:10.740031 9928 tablet_copy_service.cc:161] P 6cc5b9962ed24441bcba0c07b9f5438e: Beginning new tablet copy session on tablet 3255d89d51a94426a78364230c9b249b from peer 51e82953ed2247fa9ff24638eab3508a at {username='slave'} at 127.3.139.69:48381: session id = 51e82953ed2247fa9ff24638eab3508a-3255d89d51a94426a78364230c9b249b
I20240502 07:59:10.740861 9928 tablet_copy_source_session.cc:205] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:10.741569 14867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3255d89d51a94426a78364230c9b249b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:10.743124 14867 tablet_copy_client.cc:783] T 3255d89d51a94426a78364230c9b249b P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:10.743248 14867 tablet_copy_client.cc:647] T 3255d89d51a94426a78364230c9b249b P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:10.744166 14871 tablet_copy_client.cc:1002] T 3255d89d51a94426a78364230c9b249b P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Injecting latency into file download: 1500
I20240502 07:59:10.783982 14873 ts_tablet_manager.cc:902] T 92f37161960e4ee894c75d9034434001 P 98bef2e94ade4763a66c1ad68259744a: Initiating tablet copy from peer 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191)
I20240502 07:59:10.784384 14873 tablet_copy_client.cc:308] T 92f37161960e4ee894c75d9034434001 P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.67:43191
I20240502 07:59:10.784817 9797 tablet_copy_service.cc:140] P 1472c64688914149a318c7b8850a3df3: Received BeginTabletCopySession request for tablet 92f37161960e4ee894c75d9034434001 from peer 98bef2e94ade4763a66c1ad68259744a ({username='slave'} at 127.3.139.71:56869)
I20240502 07:59:10.784911 9797 tablet_copy_service.cc:161] P 1472c64688914149a318c7b8850a3df3: Beginning new tablet copy session on tablet 92f37161960e4ee894c75d9034434001 from peer 98bef2e94ade4763a66c1ad68259744a at {username='slave'} at 127.3.139.71:56869: session id = 98bef2e94ade4763a66c1ad68259744a-92f37161960e4ee894c75d9034434001
I20240502 07:59:10.785635 9797 tablet_copy_source_session.cc:205] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:10.786311 14873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92f37161960e4ee894c75d9034434001. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:10.787932 14873 tablet_copy_client.cc:783] T 92f37161960e4ee894c75d9034434001 P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:10.788075 14873 tablet_copy_client.cc:647] T 92f37161960e4ee894c75d9034434001 P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:10.788944 14877 tablet_copy_client.cc:1002] T 92f37161960e4ee894c75d9034434001 P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Injecting latency into file download: 1500
I20240502 07:59:12.030968 14735 tablet_copy_client.cc:523] T a12a084dba23425996e9122c083c1444 P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:12.032313 14735 tablet_bootstrap.cc:492] T a12a084dba23425996e9122c083c1444 P 98bef2e94ade4763a66c1ad68259744a: Bootstrap starting.
I20240502 07:59:12.050065 14735 tablet_bootstrap.cc:492] T a12a084dba23425996e9122c083c1444 P 98bef2e94ade4763a66c1ad68259744a: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:12.050276 14735 tablet_bootstrap.cc:492] T a12a084dba23425996e9122c083c1444 P 98bef2e94ade4763a66c1ad68259744a: Bootstrap complete.
I20240502 07:59:12.050401 14735 ts_tablet_manager.cc:1367] T a12a084dba23425996e9122c083c1444 P 98bef2e94ade4763a66c1ad68259744a: Time spent bootstrapping tablet: real 0.018s user 0.016s sys 0.000s
I20240502 07:59:12.050719 14735 raft_consensus.cc:348] T a12a084dba23425996e9122c083c1444 P 98bef2e94ade4763a66c1ad68259744a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } }
I20240502 07:59:12.050817 14735 raft_consensus.cc:729] T a12a084dba23425996e9122c083c1444 P 98bef2e94ade4763a66c1ad68259744a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 98bef2e94ade4763a66c1ad68259744a, State: Initialized, Role: LEARNER
I20240502 07:59:12.050894 14735 consensus_queue.cc:260] T a12a084dba23425996e9122c083c1444 P 98bef2e94ade4763a66c1ad68259744a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } }
I20240502 07:59:12.051172 14735 ts_tablet_manager.cc:1398] T a12a084dba23425996e9122c083c1444 P 98bef2e94ade4763a66c1ad68259744a: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240502 07:59:12.051700 9534 tablet_copy_service.cc:342] P a201db721bda4dc3a4422dc8fa9bf72e: Request end of tablet copy session 98bef2e94ade4763a66c1ad68259744a-a12a084dba23425996e9122c083c1444 received from {username='slave'} at 127.3.139.71:43779
I20240502 07:59:12.051777 9534 tablet_copy_service.cc:433] P a201db721bda4dc3a4422dc8fa9bf72e: ending tablet copy session 98bef2e94ade4763a66c1ad68259744a-a12a084dba23425996e9122c083c1444 on tablet a12a084dba23425996e9122c083c1444 with peer 98bef2e94ade4763a66c1ad68259744a
I20240502 07:59:12.105333 14804 tablet_copy_client.cc:523] T 96dab19eeaea4648913045482b3e5e9e P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:12.107511 14804 tablet_bootstrap.cc:492] T 96dab19eeaea4648913045482b3e5e9e P 09ec2b380cbf4200af90b0eee891c010: Bootstrap starting.
I20240502 07:59:12.112392 14804 log.cc:824] T 96dab19eeaea4648913045482b3e5e9e P 09ec2b380cbf4200af90b0eee891c010: Log is configured to *not* fsync() on all Append() calls
I20240502 07:59:12.124315 11520 raft_consensus.cc:1203] T a12a084dba23425996e9122c083c1444 P 98bef2e94ade4763a66c1ad68259744a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240502 07:59:12.127372 14812 tablet_copy_client.cc:523] T ee2e79e677f643409e3c8dd49b9517cf P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:12.128759 14812 tablet_bootstrap.cc:492] T ee2e79e677f643409e3c8dd49b9517cf P 8eb9945f989d4be6b51e1531a1365350: Bootstrap starting.
I20240502 07:59:12.136381 14804 tablet_bootstrap.cc:492] T 96dab19eeaea4648913045482b3e5e9e P 09ec2b380cbf4200af90b0eee891c010: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:12.136611 14804 tablet_bootstrap.cc:492] T 96dab19eeaea4648913045482b3e5e9e P 09ec2b380cbf4200af90b0eee891c010: Bootstrap complete.
I20240502 07:59:12.136857 14804 ts_tablet_manager.cc:1367] T 96dab19eeaea4648913045482b3e5e9e P 09ec2b380cbf4200af90b0eee891c010: Time spent bootstrapping tablet: real 0.029s user 0.028s sys 0.000s
I20240502 07:59:12.138818 14804 raft_consensus.cc:348] T 96dab19eeaea4648913045482b3e5e9e P 09ec2b380cbf4200af90b0eee891c010 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } }
I20240502 07:59:12.139006 14804 raft_consensus.cc:729] T 96dab19eeaea4648913045482b3e5e9e P 09ec2b380cbf4200af90b0eee891c010 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 09ec2b380cbf4200af90b0eee891c010, State: Initialized, Role: LEARNER
I20240502 07:59:12.139302 14804 consensus_queue.cc:260] T 96dab19eeaea4648913045482b3e5e9e P 09ec2b380cbf4200af90b0eee891c010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } }
I20240502 07:59:12.140103 14804 ts_tablet_manager.cc:1398] T 96dab19eeaea4648913045482b3e5e9e P 09ec2b380cbf4200af90b0eee891c010: Time spent starting tablet: real 0.003s user 0.001s sys 0.004s
I20240502 07:59:12.140789 9797 tablet_copy_service.cc:342] P 1472c64688914149a318c7b8850a3df3: Request end of tablet copy session 09ec2b380cbf4200af90b0eee891c010-96dab19eeaea4648913045482b3e5e9e received from {username='slave'} at 127.3.139.73:38883
I20240502 07:59:12.140877 9797 tablet_copy_service.cc:433] P 1472c64688914149a318c7b8850a3df3: ending tablet copy session 09ec2b380cbf4200af90b0eee891c010-96dab19eeaea4648913045482b3e5e9e on tablet 96dab19eeaea4648913045482b3e5e9e with peer 09ec2b380cbf4200af90b0eee891c010
I20240502 07:59:12.141052 14817 tablet_copy_client.cc:523] T bc896a65284c48cd864bb6b57dbe5ca0 P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:12.142649 14817 tablet_bootstrap.cc:492] T bc896a65284c48cd864bb6b57dbe5ca0 P 51e82953ed2247fa9ff24638eab3508a: Bootstrap starting.
I20240502 07:59:12.149398 14822 tablet_copy_client.cc:523] T 12e3bfe638d249c0934feda41b8a8876 P a95f1d3af33f4c879fd290ca917d6329: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:12.150521 14822 tablet_bootstrap.cc:492] T 12e3bfe638d249c0934feda41b8a8876 P a95f1d3af33f4c879fd290ca917d6329: Bootstrap starting.
I20240502 07:59:12.153584 14827 tablet_copy_client.cc:523] T d562a250cee247a48e573b25ecf00429 P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:12.154588 14827 tablet_bootstrap.cc:492] T d562a250cee247a48e573b25ecf00429 P 51e82953ed2247fa9ff24638eab3508a: Bootstrap starting.
I20240502 07:59:12.157467 14812 tablet_bootstrap.cc:492] T ee2e79e677f643409e3c8dd49b9517cf P 8eb9945f989d4be6b51e1531a1365350: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:12.157749 14812 tablet_bootstrap.cc:492] T ee2e79e677f643409e3c8dd49b9517cf P 8eb9945f989d4be6b51e1531a1365350: Bootstrap complete.
I20240502 07:59:12.157902 14812 ts_tablet_manager.cc:1367] T ee2e79e677f643409e3c8dd49b9517cf P 8eb9945f989d4be6b51e1531a1365350: Time spent bootstrapping tablet: real 0.029s user 0.025s sys 0.000s
I20240502 07:59:12.158313 14812 raft_consensus.cc:348] T ee2e79e677f643409e3c8dd49b9517cf P 8eb9945f989d4be6b51e1531a1365350 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } }
I20240502 07:59:12.158429 14812 raft_consensus.cc:729] T ee2e79e677f643409e3c8dd49b9517cf P 8eb9945f989d4be6b51e1531a1365350 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 8eb9945f989d4be6b51e1531a1365350, State: Initialized, Role: LEARNER
I20240502 07:59:12.158517 14812 consensus_queue.cc:260] T ee2e79e677f643409e3c8dd49b9517cf P 8eb9945f989d4be6b51e1531a1365350 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } }
I20240502 07:59:12.158850 14812 ts_tablet_manager.cc:1398] T ee2e79e677f643409e3c8dd49b9517cf P 8eb9945f989d4be6b51e1531a1365350: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240502 07:59:12.159763 9534 tablet_copy_service.cc:342] P a201db721bda4dc3a4422dc8fa9bf72e: Request end of tablet copy session 8eb9945f989d4be6b51e1531a1365350-ee2e79e677f643409e3c8dd49b9517cf received from {username='slave'} at 127.3.139.70:59727
I20240502 07:59:12.159844 9534 tablet_copy_service.cc:433] P a201db721bda4dc3a4422dc8fa9bf72e: ending tablet copy session 8eb9945f989d4be6b51e1531a1365350-ee2e79e677f643409e3c8dd49b9517cf on tablet ee2e79e677f643409e3c8dd49b9517cf with peer 8eb9945f989d4be6b51e1531a1365350
I20240502 07:59:12.166890 14832 tablet_copy_client.cc:523] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:12.168181 14832 tablet_bootstrap.cc:492] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 09ec2b380cbf4200af90b0eee891c010: Bootstrap starting.
I20240502 07:59:12.176893 14827 tablet_bootstrap.cc:492] T d562a250cee247a48e573b25ecf00429 P 51e82953ed2247fa9ff24638eab3508a: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:12.177196 14827 tablet_bootstrap.cc:492] T d562a250cee247a48e573b25ecf00429 P 51e82953ed2247fa9ff24638eab3508a: Bootstrap complete.
I20240502 07:59:12.177392 14827 ts_tablet_manager.cc:1367] T d562a250cee247a48e573b25ecf00429 P 51e82953ed2247fa9ff24638eab3508a: Time spent bootstrapping tablet: real 0.023s user 0.023s sys 0.000s
I20240502 07:59:12.177419 14817 tablet_bootstrap.cc:492] T bc896a65284c48cd864bb6b57dbe5ca0 P 51e82953ed2247fa9ff24638eab3508a: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:12.177714 14817 tablet_bootstrap.cc:492] T bc896a65284c48cd864bb6b57dbe5ca0 P 51e82953ed2247fa9ff24638eab3508a: Bootstrap complete.
I20240502 07:59:12.177863 14817 ts_tablet_manager.cc:1367] T bc896a65284c48cd864bb6b57dbe5ca0 P 51e82953ed2247fa9ff24638eab3508a: Time spent bootstrapping tablet: real 0.035s user 0.018s sys 0.008s
I20240502 07:59:12.177990 14827 raft_consensus.cc:348] T d562a250cee247a48e573b25ecf00429 P 51e82953ed2247fa9ff24638eab3508a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } }
I20240502 07:59:12.178215 14817 raft_consensus.cc:348] T bc896a65284c48cd864bb6b57dbe5ca0 P 51e82953ed2247fa9ff24638eab3508a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } }
I20240502 07:59:12.178314 14817 raft_consensus.cc:729] T bc896a65284c48cd864bb6b57dbe5ca0 P 51e82953ed2247fa9ff24638eab3508a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 51e82953ed2247fa9ff24638eab3508a, State: Initialized, Role: LEARNER
I20240502 07:59:12.178275 14827 raft_consensus.cc:729] T d562a250cee247a48e573b25ecf00429 P 51e82953ed2247fa9ff24638eab3508a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 51e82953ed2247fa9ff24638eab3508a, State: Initialized, Role: LEARNER
I20240502 07:59:12.178414 14817 consensus_queue.cc:260] T bc896a65284c48cd864bb6b57dbe5ca0 P 51e82953ed2247fa9ff24638eab3508a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } }
I20240502 07:59:12.178472 14827 consensus_queue.cc:260] T d562a250cee247a48e573b25ecf00429 P 51e82953ed2247fa9ff24638eab3508a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } }
I20240502 07:59:12.178718 14817 ts_tablet_manager.cc:1398] T bc896a65284c48cd864bb6b57dbe5ca0 P 51e82953ed2247fa9ff24638eab3508a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:12.178906 14827 ts_tablet_manager.cc:1398] T d562a250cee247a48e573b25ecf00429 P 51e82953ed2247fa9ff24638eab3508a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:12.179450 9666 tablet_copy_service.cc:342] P 8337af1943564c0dad18b45a597b805a: Request end of tablet copy session 51e82953ed2247fa9ff24638eab3508a-d562a250cee247a48e573b25ecf00429 received from {username='slave'} at 127.3.139.69:35563
I20240502 07:59:12.179541 9666 tablet_copy_service.cc:433] P 8337af1943564c0dad18b45a597b805a: ending tablet copy session 51e82953ed2247fa9ff24638eab3508a-d562a250cee247a48e573b25ecf00429 on tablet d562a250cee247a48e573b25ecf00429 with peer 51e82953ed2247fa9ff24638eab3508a
I20240502 07:59:12.179582 14822 tablet_bootstrap.cc:492] T 12e3bfe638d249c0934feda41b8a8876 P a95f1d3af33f4c879fd290ca917d6329: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:12.179816 14822 tablet_bootstrap.cc:492] T 12e3bfe638d249c0934feda41b8a8876 P a95f1d3af33f4c879fd290ca917d6329: Bootstrap complete.
I20240502 07:59:12.179978 14822 ts_tablet_manager.cc:1367] T 12e3bfe638d249c0934feda41b8a8876 P a95f1d3af33f4c879fd290ca917d6329: Time spent bootstrapping tablet: real 0.029s user 0.023s sys 0.000s
I20240502 07:59:12.180039 9534 tablet_copy_service.cc:342] P a201db721bda4dc3a4422dc8fa9bf72e: Request end of tablet copy session 51e82953ed2247fa9ff24638eab3508a-bc896a65284c48cd864bb6b57dbe5ca0 received from {username='slave'} at 127.3.139.69:55681
I20240502 07:59:12.180151 9534 tablet_copy_service.cc:433] P a201db721bda4dc3a4422dc8fa9bf72e: ending tablet copy session 51e82953ed2247fa9ff24638eab3508a-bc896a65284c48cd864bb6b57dbe5ca0 on tablet bc896a65284c48cd864bb6b57dbe5ca0 with peer 51e82953ed2247fa9ff24638eab3508a
I20240502 07:59:12.180531 14822 raft_consensus.cc:348] T 12e3bfe638d249c0934feda41b8a8876 P a95f1d3af33f4c879fd290ca917d6329 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } }
I20240502 07:59:12.180660 14822 raft_consensus.cc:729] T 12e3bfe638d249c0934feda41b8a8876 P a95f1d3af33f4c879fd290ca917d6329 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a95f1d3af33f4c879fd290ca917d6329, State: Initialized, Role: LEARNER
I20240502 07:59:12.181150 14822 consensus_queue.cc:260] T 12e3bfe638d249c0934feda41b8a8876 P a95f1d3af33f4c879fd290ca917d6329 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: true } }
I20240502 07:59:12.181485 14822 ts_tablet_manager.cc:1398] T 12e3bfe638d249c0934feda41b8a8876 P a95f1d3af33f4c879fd290ca917d6329: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 07:59:12.181962 9666 tablet_copy_service.cc:342] P 8337af1943564c0dad18b45a597b805a: Request end of tablet copy session a95f1d3af33f4c879fd290ca917d6329-12e3bfe638d249c0934feda41b8a8876 received from {username='slave'} at 127.3.139.72:42289
I20240502 07:59:12.182029 9666 tablet_copy_service.cc:433] P 8337af1943564c0dad18b45a597b805a: ending tablet copy session a95f1d3af33f4c879fd290ca917d6329-12e3bfe638d249c0934feda41b8a8876 on tablet 12e3bfe638d249c0934feda41b8a8876 with peer a95f1d3af33f4c879fd290ca917d6329
I20240502 07:59:12.197099 11260 raft_consensus.cc:1203] T bc896a65284c48cd864bb6b57dbe5ca0 P 51e82953ed2247fa9ff24638eab3508a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240502 07:59:12.197952 14832 tablet_bootstrap.cc:492] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 09ec2b380cbf4200af90b0eee891c010: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:12.198211 14832 tablet_bootstrap.cc:492] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 09ec2b380cbf4200af90b0eee891c010: Bootstrap complete.
I20240502 07:59:12.198370 14832 ts_tablet_manager.cc:1367] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 09ec2b380cbf4200af90b0eee891c010: Time spent bootstrapping tablet: real 0.030s user 0.022s sys 0.000s
I20240502 07:59:12.198879 14832 raft_consensus.cc:348] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 09ec2b380cbf4200af90b0eee891c010 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } }
I20240502 07:59:12.199000 14832 raft_consensus.cc:729] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 09ec2b380cbf4200af90b0eee891c010 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 09ec2b380cbf4200af90b0eee891c010, State: Initialized, Role: LEARNER
I20240502 07:59:12.199123 14832 consensus_queue.cc:260] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 09ec2b380cbf4200af90b0eee891c010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } }
I20240502 07:59:12.199458 14832 ts_tablet_manager.cc:1398] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 09ec2b380cbf4200af90b0eee891c010: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:12.200066 9928 tablet_copy_service.cc:342] P 6cc5b9962ed24441bcba0c07b9f5438e: Request end of tablet copy session 09ec2b380cbf4200af90b0eee891c010-7da9d3b7fe8c40f6a68bb1b7a926d994 received from {username='slave'} at 127.3.139.73:40787
I20240502 07:59:12.200141 9928 tablet_copy_service.cc:433] P 6cc5b9962ed24441bcba0c07b9f5438e: ending tablet copy session 09ec2b380cbf4200af90b0eee891c010-7da9d3b7fe8c40f6a68bb1b7a926d994 on tablet 7da9d3b7fe8c40f6a68bb1b7a926d994 with peer 09ec2b380cbf4200af90b0eee891c010
I20240502 07:59:12.202909 14837 tablet_copy_client.cc:523] T 265eca6fff644d83857ed48a04864a58 P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:12.204237 14837 tablet_bootstrap.cc:492] T 265eca6fff644d83857ed48a04864a58 P 98bef2e94ade4763a66c1ad68259744a: Bootstrap starting.
I20240502 07:59:12.206892 14842 tablet_copy_client.cc:523] T 364d227472894712a2ae15f4589a5a5b P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:12.208129 14842 tablet_bootstrap.cc:492] T 364d227472894712a2ae15f4589a5a5b P 09ec2b380cbf4200af90b0eee891c010: Bootstrap starting.
I20240502 07:59:12.217375 14852 tablet_copy_client.cc:523] T 50f059668c6b4205a15d2c076e2789b0 P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:12.217499 14847 tablet_copy_client.cc:523] T d37b8a7f016647a7a03baedc37ed5419 P 8eb9945f989d4be6b51e1531a1365350: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:12.218528 14852 tablet_bootstrap.cc:492] T 50f059668c6b4205a15d2c076e2789b0 P 09ec2b380cbf4200af90b0eee891c010: Bootstrap starting.
I20240502 07:59:12.218712 14847 tablet_bootstrap.cc:492] T d37b8a7f016647a7a03baedc37ed5419 P 8eb9945f989d4be6b51e1531a1365350: Bootstrap starting.
I20240502 07:59:12.225304 14837 tablet_bootstrap.cc:492] T 265eca6fff644d83857ed48a04864a58 P 98bef2e94ade4763a66c1ad68259744a: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:12.225550 14837 tablet_bootstrap.cc:492] T 265eca6fff644d83857ed48a04864a58 P 98bef2e94ade4763a66c1ad68259744a: Bootstrap complete.
I20240502 07:59:12.225703 14837 ts_tablet_manager.cc:1367] T 265eca6fff644d83857ed48a04864a58 P 98bef2e94ade4763a66c1ad68259744a: Time spent bootstrapping tablet: real 0.022s user 0.012s sys 0.007s
I20240502 07:59:12.226068 14837 raft_consensus.cc:348] T 265eca6fff644d83857ed48a04864a58 P 98bef2e94ade4763a66c1ad68259744a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } }
I20240502 07:59:12.226181 14837 raft_consensus.cc:729] T 265eca6fff644d83857ed48a04864a58 P 98bef2e94ade4763a66c1ad68259744a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 98bef2e94ade4763a66c1ad68259744a, State: Initialized, Role: LEARNER
I20240502 07:59:12.226262 14837 consensus_queue.cc:260] T 265eca6fff644d83857ed48a04864a58 P 98bef2e94ade4763a66c1ad68259744a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } }
I20240502 07:59:12.226624 14837 ts_tablet_manager.cc:1398] T 265eca6fff644d83857ed48a04864a58 P 98bef2e94ade4763a66c1ad68259744a: Time spent starting tablet: real 0.001s user 0.003s sys 0.001s
I20240502 07:59:12.227145 9534 tablet_copy_service.cc:342] P a201db721bda4dc3a4422dc8fa9bf72e: Request end of tablet copy session 98bef2e94ade4763a66c1ad68259744a-265eca6fff644d83857ed48a04864a58 received from {username='slave'} at 127.3.139.71:43779
I20240502 07:59:12.227221 9534 tablet_copy_service.cc:433] P a201db721bda4dc3a4422dc8fa9bf72e: ending tablet copy session 98bef2e94ade4763a66c1ad68259744a-265eca6fff644d83857ed48a04864a58 on tablet 265eca6fff644d83857ed48a04864a58 with peer 98bef2e94ade4763a66c1ad68259744a
I20240502 07:59:12.228780 14857 tablet_copy_client.cc:523] T 0e4efe560ef74763b80b70665c11d7f1 P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:12.229895 14857 tablet_bootstrap.cc:492] T 0e4efe560ef74763b80b70665c11d7f1 P 09ec2b380cbf4200af90b0eee891c010: Bootstrap starting.
I20240502 07:59:12.235523 14842 tablet_bootstrap.cc:492] T 364d227472894712a2ae15f4589a5a5b P 09ec2b380cbf4200af90b0eee891c010: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:12.235824 14842 tablet_bootstrap.cc:492] T 364d227472894712a2ae15f4589a5a5b P 09ec2b380cbf4200af90b0eee891c010: Bootstrap complete.
I20240502 07:59:12.236007 14842 ts_tablet_manager.cc:1367] T 364d227472894712a2ae15f4589a5a5b P 09ec2b380cbf4200af90b0eee891c010: Time spent bootstrapping tablet: real 0.028s user 0.022s sys 0.002s
I20240502 07:59:12.236416 14842 raft_consensus.cc:348] T 364d227472894712a2ae15f4589a5a5b P 09ec2b380cbf4200af90b0eee891c010 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } }
I20240502 07:59:12.236578 14842 raft_consensus.cc:729] T 364d227472894712a2ae15f4589a5a5b P 09ec2b380cbf4200af90b0eee891c010 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 09ec2b380cbf4200af90b0eee891c010, State: Initialized, Role: LEARNER
I20240502 07:59:12.236737 14842 consensus_queue.cc:260] T 364d227472894712a2ae15f4589a5a5b P 09ec2b380cbf4200af90b0eee891c010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } }
I20240502 07:59:12.237435 14842 ts_tablet_manager.cc:1398] T 364d227472894712a2ae15f4589a5a5b P 09ec2b380cbf4200af90b0eee891c010: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:12.237891 9666 tablet_copy_service.cc:342] P 8337af1943564c0dad18b45a597b805a: Request end of tablet copy session 09ec2b380cbf4200af90b0eee891c010-364d227472894712a2ae15f4589a5a5b received from {username='slave'} at 127.3.139.73:58997
I20240502 07:59:12.237960 9666 tablet_copy_service.cc:433] P 8337af1943564c0dad18b45a597b805a: ending tablet copy session 09ec2b380cbf4200af90b0eee891c010-364d227472894712a2ae15f4589a5a5b on tablet 364d227472894712a2ae15f4589a5a5b with peer 09ec2b380cbf4200af90b0eee891c010
I20240502 07:59:12.238502 14862 tablet_copy_client.cc:523] T 8ceb129e9cbe4c469e7dfba419719556 P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:12.239698 14862 tablet_bootstrap.cc:492] T 8ceb129e9cbe4c469e7dfba419719556 P 98bef2e94ade4763a66c1ad68259744a: Bootstrap starting.
I20240502 07:59:12.243804 14852 tablet_bootstrap.cc:492] T 50f059668c6b4205a15d2c076e2789b0 P 09ec2b380cbf4200af90b0eee891c010: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:12.244099 14852 tablet_bootstrap.cc:492] T 50f059668c6b4205a15d2c076e2789b0 P 09ec2b380cbf4200af90b0eee891c010: Bootstrap complete.
I20240502 07:59:12.244388 14867 tablet_copy_client.cc:523] T 3255d89d51a94426a78364230c9b249b P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:12.245630 14867 tablet_bootstrap.cc:492] T 3255d89d51a94426a78364230c9b249b P 51e82953ed2247fa9ff24638eab3508a: Bootstrap starting.
I20240502 07:59:12.249316 14852 ts_tablet_manager.cc:1367] T 50f059668c6b4205a15d2c076e2789b0 P 09ec2b380cbf4200af90b0eee891c010: Time spent bootstrapping tablet: real 0.031s user 0.025s sys 0.000s
I20240502 07:59:12.249760 14852 raft_consensus.cc:348] T 50f059668c6b4205a15d2c076e2789b0 P 09ec2b380cbf4200af90b0eee891c010 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } }
I20240502 07:59:12.249917 14852 raft_consensus.cc:729] T 50f059668c6b4205a15d2c076e2789b0 P 09ec2b380cbf4200af90b0eee891c010 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 09ec2b380cbf4200af90b0eee891c010, State: Initialized, Role: LEARNER
I20240502 07:59:12.250037 14852 consensus_queue.cc:260] T 50f059668c6b4205a15d2c076e2789b0 P 09ec2b380cbf4200af90b0eee891c010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } }
I20240502 07:59:12.250418 14852 ts_tablet_manager.cc:1398] T 50f059668c6b4205a15d2c076e2789b0 P 09ec2b380cbf4200af90b0eee891c010: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:12.250864 9666 tablet_copy_service.cc:342] P 8337af1943564c0dad18b45a597b805a: Request end of tablet copy session 09ec2b380cbf4200af90b0eee891c010-50f059668c6b4205a15d2c076e2789b0 received from {username='slave'} at 127.3.139.73:58997
I20240502 07:59:12.250936 9666 tablet_copy_service.cc:433] P 8337af1943564c0dad18b45a597b805a: ending tablet copy session 09ec2b380cbf4200af90b0eee891c010-50f059668c6b4205a15d2c076e2789b0 on tablet 50f059668c6b4205a15d2c076e2789b0 with peer 09ec2b380cbf4200af90b0eee891c010
I20240502 07:59:12.258165 14857 tablet_bootstrap.cc:492] T 0e4efe560ef74763b80b70665c11d7f1 P 09ec2b380cbf4200af90b0eee891c010: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:12.258318 11654 raft_consensus.cc:1203] T 12e3bfe638d249c0934feda41b8a8876 P a95f1d3af33f4c879fd290ca917d6329 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240502 07:59:12.258455 14857 tablet_bootstrap.cc:492] T 0e4efe560ef74763b80b70665c11d7f1 P 09ec2b380cbf4200af90b0eee891c010: Bootstrap complete.
I20240502 07:59:12.259071 14857 ts_tablet_manager.cc:1367] T 0e4efe560ef74763b80b70665c11d7f1 P 09ec2b380cbf4200af90b0eee891c010: Time spent bootstrapping tablet: real 0.029s user 0.023s sys 0.004s
I20240502 07:59:12.259531 14857 raft_consensus.cc:348] T 0e4efe560ef74763b80b70665c11d7f1 P 09ec2b380cbf4200af90b0eee891c010 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } }
I20240502 07:59:12.259667 14857 raft_consensus.cc:729] T 0e4efe560ef74763b80b70665c11d7f1 P 09ec2b380cbf4200af90b0eee891c010 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 09ec2b380cbf4200af90b0eee891c010, State: Initialized, Role: LEARNER
I20240502 07:59:12.259783 14857 consensus_queue.cc:260] T 0e4efe560ef74763b80b70665c11d7f1 P 09ec2b380cbf4200af90b0eee891c010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } }
I20240502 07:59:12.260152 14857 ts_tablet_manager.cc:1398] T 0e4efe560ef74763b80b70665c11d7f1 P 09ec2b380cbf4200af90b0eee891c010: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:12.260592 9797 tablet_copy_service.cc:342] P 1472c64688914149a318c7b8850a3df3: Request end of tablet copy session 09ec2b380cbf4200af90b0eee891c010-0e4efe560ef74763b80b70665c11d7f1 received from {username='slave'} at 127.3.139.73:38883
I20240502 07:59:12.260668 9797 tablet_copy_service.cc:433] P 1472c64688914149a318c7b8850a3df3: ending tablet copy session 09ec2b380cbf4200af90b0eee891c010-0e4efe560ef74763b80b70665c11d7f1 on tablet 0e4efe560ef74763b80b70665c11d7f1 with peer 09ec2b380cbf4200af90b0eee891c010
I20240502 07:59:12.273182 14862 tablet_bootstrap.cc:492] T 8ceb129e9cbe4c469e7dfba419719556 P 98bef2e94ade4763a66c1ad68259744a: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:12.273480 14862 tablet_bootstrap.cc:492] T 8ceb129e9cbe4c469e7dfba419719556 P 98bef2e94ade4763a66c1ad68259744a: Bootstrap complete.
I20240502 07:59:12.273669 14862 ts_tablet_manager.cc:1367] T 8ceb129e9cbe4c469e7dfba419719556 P 98bef2e94ade4763a66c1ad68259744a: Time spent bootstrapping tablet: real 0.034s user 0.025s sys 0.000s
I20240502 07:59:12.274093 14862 raft_consensus.cc:348] T 8ceb129e9cbe4c469e7dfba419719556 P 98bef2e94ade4763a66c1ad68259744a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } }
I20240502 07:59:12.274258 14862 raft_consensus.cc:729] T 8ceb129e9cbe4c469e7dfba419719556 P 98bef2e94ade4763a66c1ad68259744a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 98bef2e94ade4763a66c1ad68259744a, State: Initialized, Role: LEARNER
I20240502 07:59:12.274360 14867 tablet_bootstrap.cc:492] T 3255d89d51a94426a78364230c9b249b P 51e82953ed2247fa9ff24638eab3508a: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:12.274384 14862 consensus_queue.cc:260] T 8ceb129e9cbe4c469e7dfba419719556 P 98bef2e94ade4763a66c1ad68259744a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } }
I20240502 07:59:12.274750 14867 tablet_bootstrap.cc:492] T 3255d89d51a94426a78364230c9b249b P 51e82953ed2247fa9ff24638eab3508a: Bootstrap complete.
I20240502 07:59:12.274751 14862 ts_tablet_manager.cc:1398] T 8ceb129e9cbe4c469e7dfba419719556 P 98bef2e94ade4763a66c1ad68259744a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:12.274943 14867 ts_tablet_manager.cc:1367] T 3255d89d51a94426a78364230c9b249b P 51e82953ed2247fa9ff24638eab3508a: Time spent bootstrapping tablet: real 0.029s user 0.025s sys 0.000s
I20240502 07:59:12.275242 9666 tablet_copy_service.cc:342] P 8337af1943564c0dad18b45a597b805a: Request end of tablet copy session 98bef2e94ade4763a66c1ad68259744a-8ceb129e9cbe4c469e7dfba419719556 received from {username='slave'} at 127.3.139.71:52353
I20240502 07:59:12.275310 9666 tablet_copy_service.cc:433] P 8337af1943564c0dad18b45a597b805a: ending tablet copy session 98bef2e94ade4763a66c1ad68259744a-8ceb129e9cbe4c469e7dfba419719556 on tablet 8ceb129e9cbe4c469e7dfba419719556 with peer 98bef2e94ade4763a66c1ad68259744a
I20240502 07:59:12.275416 14867 raft_consensus.cc:348] T 3255d89d51a94426a78364230c9b249b P 51e82953ed2247fa9ff24638eab3508a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } }
I20240502 07:59:12.275557 14867 raft_consensus.cc:729] T 3255d89d51a94426a78364230c9b249b P 51e82953ed2247fa9ff24638eab3508a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 51e82953ed2247fa9ff24638eab3508a, State: Initialized, Role: LEARNER
I20240502 07:59:12.275676 14867 consensus_queue.cc:260] T 3255d89d51a94426a78364230c9b249b P 51e82953ed2247fa9ff24638eab3508a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } }
I20240502 07:59:12.275974 14867 ts_tablet_manager.cc:1398] T 3255d89d51a94426a78364230c9b249b P 51e82953ed2247fa9ff24638eab3508a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:12.276062 14847 tablet_bootstrap.cc:492] T d37b8a7f016647a7a03baedc37ed5419 P 8eb9945f989d4be6b51e1531a1365350: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:12.276275 14847 tablet_bootstrap.cc:492] T d37b8a7f016647a7a03baedc37ed5419 P 8eb9945f989d4be6b51e1531a1365350: Bootstrap complete.
I20240502 07:59:12.276428 14847 ts_tablet_manager.cc:1367] T d37b8a7f016647a7a03baedc37ed5419 P 8eb9945f989d4be6b51e1531a1365350: Time spent bootstrapping tablet: real 0.058s user 0.026s sys 0.002s
I20240502 07:59:12.276479 9928 tablet_copy_service.cc:342] P 6cc5b9962ed24441bcba0c07b9f5438e: Request end of tablet copy session 51e82953ed2247fa9ff24638eab3508a-3255d89d51a94426a78364230c9b249b received from {username='slave'} at 127.3.139.69:48381
I20240502 07:59:12.276557 9928 tablet_copy_service.cc:433] P 6cc5b9962ed24441bcba0c07b9f5438e: ending tablet copy session 51e82953ed2247fa9ff24638eab3508a-3255d89d51a94426a78364230c9b249b on tablet 3255d89d51a94426a78364230c9b249b with peer 51e82953ed2247fa9ff24638eab3508a
I20240502 07:59:12.276829 14847 raft_consensus.cc:348] T d37b8a7f016647a7a03baedc37ed5419 P 8eb9945f989d4be6b51e1531a1365350 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } }
I20240502 07:59:12.276948 14847 raft_consensus.cc:729] T d37b8a7f016647a7a03baedc37ed5419 P 8eb9945f989d4be6b51e1531a1365350 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 8eb9945f989d4be6b51e1531a1365350, State: Initialized, Role: LEARNER
I20240502 07:59:12.277051 14847 consensus_queue.cc:260] T d37b8a7f016647a7a03baedc37ed5419 P 8eb9945f989d4be6b51e1531a1365350 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: true } }
I20240502 07:59:12.277321 14847 ts_tablet_manager.cc:1398] T d37b8a7f016647a7a03baedc37ed5419 P 8eb9945f989d4be6b51e1531a1365350: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:12.277794 9797 tablet_copy_service.cc:342] P 1472c64688914149a318c7b8850a3df3: Request end of tablet copy session 8eb9945f989d4be6b51e1531a1365350-d37b8a7f016647a7a03baedc37ed5419 received from {username='slave'} at 127.3.139.70:41973
I20240502 07:59:12.277871 9797 tablet_copy_service.cc:433] P 1472c64688914149a318c7b8850a3df3: ending tablet copy session 8eb9945f989d4be6b51e1531a1365350-d37b8a7f016647a7a03baedc37ed5419 on tablet d37b8a7f016647a7a03baedc37ed5419 with peer 8eb9945f989d4be6b51e1531a1365350
I20240502 07:59:12.289207 14873 tablet_copy_client.cc:523] T 92f37161960e4ee894c75d9034434001 P 98bef2e94ade4763a66c1ad68259744a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:12.290390 14873 tablet_bootstrap.cc:492] T 92f37161960e4ee894c75d9034434001 P 98bef2e94ade4763a66c1ad68259744a: Bootstrap starting.
I20240502 07:59:12.311573 14873 tablet_bootstrap.cc:492] T 92f37161960e4ee894c75d9034434001 P 98bef2e94ade4763a66c1ad68259744a: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:12.311797 14873 tablet_bootstrap.cc:492] T 92f37161960e4ee894c75d9034434001 P 98bef2e94ade4763a66c1ad68259744a: Bootstrap complete.
I20240502 07:59:12.311985 14873 ts_tablet_manager.cc:1367] T 92f37161960e4ee894c75d9034434001 P 98bef2e94ade4763a66c1ad68259744a: Time spent bootstrapping tablet: real 0.022s user 0.019s sys 0.004s
I20240502 07:59:12.312361 14873 raft_consensus.cc:348] T 92f37161960e4ee894c75d9034434001 P 98bef2e94ade4763a66c1ad68259744a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } }
I20240502 07:59:12.312494 14873 raft_consensus.cc:729] T 92f37161960e4ee894c75d9034434001 P 98bef2e94ade4763a66c1ad68259744a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 98bef2e94ade4763a66c1ad68259744a, State: Initialized, Role: LEARNER
I20240502 07:59:12.312633 14873 consensus_queue.cc:260] T 92f37161960e4ee894c75d9034434001 P 98bef2e94ade4763a66c1ad68259744a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: NON_VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: true } }
I20240502 07:59:12.313016 14873 ts_tablet_manager.cc:1398] T 92f37161960e4ee894c75d9034434001 P 98bef2e94ade4763a66c1ad68259744a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:12.313436 9797 tablet_copy_service.cc:342] P 1472c64688914149a318c7b8850a3df3: Request end of tablet copy session 98bef2e94ade4763a66c1ad68259744a-92f37161960e4ee894c75d9034434001 received from {username='slave'} at 127.3.139.71:56869
I20240502 07:59:12.313508 9797 tablet_copy_service.cc:433] P 1472c64688914149a318c7b8850a3df3: ending tablet copy session 98bef2e94ade4763a66c1ad68259744a-92f37161960e4ee894c75d9034434001 on tablet 92f37161960e4ee894c75d9034434001 with peer 98bef2e94ade4763a66c1ad68259744a
I20240502 07:59:12.364955 12589 raft_consensus.cc:1203] T 364d227472894712a2ae15f4589a5a5b P 09ec2b380cbf4200af90b0eee891c010 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240502 07:59:12.369249 11520 raft_consensus.cc:1203] T 92f37161960e4ee894c75d9034434001 P 98bef2e94ade4763a66c1ad68259744a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240502 07:59:12.407408 12589 raft_consensus.cc:1203] T 50f059668c6b4205a15d2c076e2789b0 P 09ec2b380cbf4200af90b0eee891c010 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240502 07:59:12.437394 12589 raft_consensus.cc:1203] T 0e4efe560ef74763b80b70665c11d7f1 P 09ec2b380cbf4200af90b0eee891c010 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240502 07:59:12.491887 11389 raft_consensus.cc:1203] T ee2e79e677f643409e3c8dd49b9517cf P 8eb9945f989d4be6b51e1531a1365350 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240502 07:59:12.598285 11260 raft_consensus.cc:1203] T d562a250cee247a48e573b25ecf00429 P 51e82953ed2247fa9ff24638eab3508a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240502 07:59:12.607375 12589 raft_consensus.cc:1203] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 09ec2b380cbf4200af90b0eee891c010 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240502 07:59:12.625345 15356 raft_consensus.cc:1050] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e: attempting to promote NON_VOTER 98bef2e94ade4763a66c1ad68259744a to VOTER
I20240502 07:59:12.625732 15356 consensus_queue.cc:237] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } }
I20240502 07:59:12.627101 9908 raft_consensus.cc:1261] T a12a084dba23425996e9122c083c1444 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:12.627120 11260 raft_consensus.cc:1261] T a12a084dba23425996e9122c083c1444 P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:12.627140 11520 raft_consensus.cc:1261] T a12a084dba23425996e9122c083c1444 P 98bef2e94ade4763a66c1ad68259744a [term 1 LEARNER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:12.627516 15432 consensus_queue.cc:1035] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:12.627715 14956 consensus_queue.cc:1035] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:12.627880 15434 consensus_queue.cc:1035] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:12.629698 15432 raft_consensus.cc:2932] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:12.630136 9908 raft_consensus.cc:2932] T a12a084dba23425996e9122c083c1444 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:12.630349 11520 raft_consensus.cc:2932] T a12a084dba23425996e9122c083c1444 P 98bef2e94ade4763a66c1ad68259744a [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:12.631628 11260 raft_consensus.cc:2932] T a12a084dba23425996e9122c083c1444 P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:12.632916 9370 catalog_manager.cc:5484] T a12a084dba23425996e9122c083c1444 P 98bef2e94ade4763a66c1ad68259744a reported cstate change: config changed from index 7 to 8, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:12.636456 9515 consensus_queue.cc:237] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } }
I20240502 07:59:12.637571 11520 raft_consensus.cc:1261] T a12a084dba23425996e9122c083c1444 P 98bef2e94ade4763a66c1ad68259744a [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:12.637816 11260 raft_consensus.cc:1261] T a12a084dba23425996e9122c083c1444 P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:12.637912 15434 consensus_queue.cc:1035] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:12.638139 15432 consensus_queue.cc:1035] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:12.639613 15356 raft_consensus.cc:2932] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:12.640097 11520 raft_consensus.cc:2932] T a12a084dba23425996e9122c083c1444 P 98bef2e94ade4763a66c1ad68259744a [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:12.641156 9360 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet a12a084dba23425996e9122c083c1444 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:12.641561 11259 raft_consensus.cc:2932] T a12a084dba23425996e9122c083c1444 P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:12.642144 9367 catalog_manager.cc:5484] T a12a084dba23425996e9122c083c1444 P a201db721bda4dc3a4422dc8fa9bf72e reported cstate change: config changed from index 8 to 9, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:12.645402 9886 tablet_service.cc:1514] Processing DeleteTablet for tablet a12a084dba23425996e9122c083c1444 with delete_type TABLET_DATA_TOMBSTONED (TS 6cc5b9962ed24441bcba0c07b9f5438e not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:41940
I20240502 07:59:12.645758 15442 tablet_replica.cc:330] T a12a084dba23425996e9122c083c1444 P 6cc5b9962ed24441bcba0c07b9f5438e: stopping tablet replica
I20240502 07:59:12.645887 15442 raft_consensus.cc:2229] T a12a084dba23425996e9122c083c1444 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:12.646078 15442 raft_consensus.cc:2258] T a12a084dba23425996e9122c083c1444 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:12.646682 15442 ts_tablet_manager.cc:1875] T a12a084dba23425996e9122c083c1444 P 6cc5b9962ed24441bcba0c07b9f5438e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:12.648456 15442 ts_tablet_manager.cc:1888] T a12a084dba23425996e9122c083c1444 P 6cc5b9962ed24441bcba0c07b9f5438e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240502 07:59:12.648519 15442 log.cc:1196] T a12a084dba23425996e9122c083c1444 P 6cc5b9962ed24441bcba0c07b9f5438e: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal/wals/a12a084dba23425996e9122c083c1444
I20240502 07:59:12.648985 9358 catalog_manager.cc:4830] TS 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379): tablet a12a084dba23425996e9122c083c1444 (table rebalance_test_table_3 [id=47c27ad65ee1405898b667c2642e1b3a]) successfully deleted
I20240502 07:59:12.651397 12589 raft_consensus.cc:1203] T 96dab19eeaea4648913045482b3e5e9e P 09ec2b380cbf4200af90b0eee891c010 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:12.655189 11520 raft_consensus.cc:1203] T 265eca6fff644d83857ed48a04864a58 P 98bef2e94ade4763a66c1ad68259744a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240502 07:59:12.669224 15434 raft_consensus.cc:1050] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e: attempting to promote NON_VOTER 51e82953ed2247fa9ff24638eab3508a to VOTER
I20240502 07:59:12.669576 15434 consensus_queue.cc:237] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } }
I20240502 07:59:12.671121 11259 raft_consensus.cc:1261] T bc896a65284c48cd864bb6b57dbe5ca0 P 51e82953ed2247fa9ff24638eab3508a [term 1 LEARNER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:12.671176 9646 raft_consensus.cc:1261] T bc896a65284c48cd864bb6b57dbe5ca0 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:12.671200 11654 raft_consensus.cc:1261] T bc896a65284c48cd864bb6b57dbe5ca0 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:12.671564 14956 consensus_queue.cc:1035] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:12.671747 15432 consensus_queue.cc:1035] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:12.671929 15434 consensus_queue.cc:1035] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:12.673671 15434 raft_consensus.cc:2932] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:12.674126 9646 raft_consensus.cc:2932] T bc896a65284c48cd864bb6b57dbe5ca0 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:12.674477 11259 raft_consensus.cc:2932] T bc896a65284c48cd864bb6b57dbe5ca0 P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:12.676394 9372 catalog_manager.cc:5484] T bc896a65284c48cd864bb6b57dbe5ca0 P 8337af1943564c0dad18b45a597b805a reported cstate change: config changed from index 7 to 8, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:12.677196 11654 raft_consensus.cc:2932] T bc896a65284c48cd864bb6b57dbe5ca0 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:12.680317 9515 consensus_queue.cc:237] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } }
I20240502 07:59:12.681267 11259 raft_consensus.cc:1261] T bc896a65284c48cd864bb6b57dbe5ca0 P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:12.681450 11654 raft_consensus.cc:1261] T bc896a65284c48cd864bb6b57dbe5ca0 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:12.681607 15434 consensus_queue.cc:1035] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:12.681777 14956 consensus_queue.cc:1035] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:12.683182 14956 raft_consensus.cc:2932] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:12.683635 11259 raft_consensus.cc:2932] T bc896a65284c48cd864bb6b57dbe5ca0 P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:12.683869 11654 raft_consensus.cc:2932] T bc896a65284c48cd864bb6b57dbe5ca0 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:12.684485 9360 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet bc896a65284c48cd864bb6b57dbe5ca0 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:12.685530 9370 catalog_manager.cc:5484] T bc896a65284c48cd864bb6b57dbe5ca0 P a201db721bda4dc3a4422dc8fa9bf72e reported cstate change: config changed from index 8 to 9, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:12.688012 9624 tablet_service.cc:1514] Processing DeleteTablet for tablet bc896a65284c48cd864bb6b57dbe5ca0 with delete_type TABLET_DATA_TOMBSTONED (TS 8337af1943564c0dad18b45a597b805a not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:53862
I20240502 07:59:12.688336 15447 tablet_replica.cc:330] T bc896a65284c48cd864bb6b57dbe5ca0 P 8337af1943564c0dad18b45a597b805a: stopping tablet replica
I20240502 07:59:12.688463 15447 raft_consensus.cc:2229] T bc896a65284c48cd864bb6b57dbe5ca0 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:12.688619 15447 raft_consensus.cc:2258] T bc896a65284c48cd864bb6b57dbe5ca0 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:12.689131 15447 ts_tablet_manager.cc:1875] T bc896a65284c48cd864bb6b57dbe5ca0 P 8337af1943564c0dad18b45a597b805a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:12.690747 15447 ts_tablet_manager.cc:1888] T bc896a65284c48cd864bb6b57dbe5ca0 P 8337af1943564c0dad18b45a597b805a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240502 07:59:12.690810 15447 log.cc:1196] T bc896a65284c48cd864bb6b57dbe5ca0 P 8337af1943564c0dad18b45a597b805a: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal/wals/bc896a65284c48cd864bb6b57dbe5ca0
I20240502 07:59:12.691296 9358 catalog_manager.cc:4830] TS 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667): tablet bc896a65284c48cd864bb6b57dbe5ca0 (table rebalance_test_table_8 [id=6aecdf32373e4c9db62c5a4844dc38ca]) successfully deleted
I20240502 07:59:12.733292 11520 raft_consensus.cc:1203] T 8ceb129e9cbe4c469e7dfba419719556 P 98bef2e94ade4763a66c1ad68259744a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240502 07:59:12.814734 14872 raft_consensus.cc:1050] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3: attempting to promote NON_VOTER 98bef2e94ade4763a66c1ad68259744a to VOTER
I20240502 07:59:12.815200 14872 consensus_queue.cc:237] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } }
I20240502 07:59:12.816705 11389 raft_consensus.cc:1261] T 92f37161960e4ee894c75d9034434001 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:12.816838 9908 raft_consensus.cc:1261] T 92f37161960e4ee894c75d9034434001 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:12.817036 11520 raft_consensus.cc:1261] T 92f37161960e4ee894c75d9034434001 P 98bef2e94ade4763a66c1ad68259744a [term 1 LEARNER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:12.817248 15460 consensus_queue.cc:1035] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:12.817422 15453 consensus_queue.cc:1035] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:12.817669 15461 consensus_queue.cc:1035] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:12.820726 15461 raft_consensus.cc:2932] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:12.821246 9908 raft_consensus.cc:2932] T 92f37161960e4ee894c75d9034434001 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:12.821549 11389 raft_consensus.cc:2932] T 92f37161960e4ee894c75d9034434001 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:12.822705 11520 raft_consensus.cc:2932] T 92f37161960e4ee894c75d9034434001 P 98bef2e94ade4763a66c1ad68259744a [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:12.823884 9369 catalog_manager.cc:5484] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 reported cstate change: config changed from index 7 to 8, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1472c64688914149a318c7b8850a3df3" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:12.832274 9777 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "92f37161960e4ee894c75d9034434001"
dest_uuid: "1472c64688914149a318c7b8850a3df3"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:49276
I20240502 07:59:12.832406 9777 raft_consensus.cc:595] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Received request to transfer leadership
I20240502 07:59:12.839946 15481 raft_consensus.cc:1050] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a: attempting to promote NON_VOTER a95f1d3af33f4c879fd290ca917d6329 to VOTER
I20240502 07:59:12.840493 15481 consensus_queue.cc:237] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } }
I20240502 07:59:12.841706 11654 raft_consensus.cc:1261] T 12e3bfe638d249c0934feda41b8a8876 P a95f1d3af33f4c879fd290ca917d6329 [term 1 LEARNER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:12.841702 11389 raft_consensus.cc:1261] T 12e3bfe638d249c0934feda41b8a8876 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:12.842015 9515 raft_consensus.cc:1261] T 12e3bfe638d249c0934feda41b8a8876 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:12.842417 15481 consensus_queue.cc:1035] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:12.842629 15013 consensus_queue.cc:1035] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:12.842876 15480 consensus_queue.cc:1035] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:12.844491 15480 raft_consensus.cc:2932] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:12.844950 11654 raft_consensus.cc:2932] T 12e3bfe638d249c0934feda41b8a8876 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:12.845341 11389 raft_consensus.cc:2932] T 12e3bfe638d249c0934feda41b8a8876 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:12.846482 9515 raft_consensus.cc:2932] T 12e3bfe638d249c0934feda41b8a8876 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:12.846997 9370 catalog_manager.cc:5484] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a reported cstate change: config changed from index 7 to 8, a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:12.865172 11259 raft_consensus.cc:1203] T 3255d89d51a94426a78364230c9b249b P 51e82953ed2247fa9ff24638eab3508a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240502 07:59:12.867345 11389 raft_consensus.cc:1203] T d37b8a7f016647a7a03baedc37ed5419 P 8eb9945f989d4be6b51e1531a1365350 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240502 07:59:12.878239 9646 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "12e3bfe638d249c0934feda41b8a8876"
dest_uuid: "8337af1943564c0dad18b45a597b805a"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:38122
I20240502 07:59:12.878346 9646 raft_consensus.cc:595] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Received request to transfer leadership
I20240502 07:59:12.889504 15480 raft_consensus.cc:1050] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a: attempting to promote NON_VOTER 09ec2b380cbf4200af90b0eee891c010 to VOTER
I20240502 07:59:12.889675 15532 raft_consensus.cc:1050] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a: attempting to promote NON_VOTER 09ec2b380cbf4200af90b0eee891c010 to VOTER
I20240502 07:59:12.889904 15480 consensus_queue.cc:237] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } }
I20240502 07:59:12.890044 15532 consensus_queue.cc:237] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } }
I20240502 07:59:12.891470 9515 raft_consensus.cc:1261] T 364d227472894712a2ae15f4589a5a5b P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:12.891646 11520 raft_consensus.cc:1261] T 364d227472894712a2ae15f4589a5a5b P 98bef2e94ade4763a66c1ad68259744a [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:12.891750 12588 raft_consensus.cc:1261] T 364d227472894712a2ae15f4589a5a5b P 09ec2b380cbf4200af90b0eee891c010 [term 1 LEARNER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:12.892091 12589 raft_consensus.cc:1261] T 50f059668c6b4205a15d2c076e2789b0 P 09ec2b380cbf4200af90b0eee891c010 [term 1 LEARNER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:12.892097 15535 consensus_queue.cc:1035] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:12.892565 15013 consensus_queue.cc:1035] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:12.892699 15481 consensus_queue.cc:1035] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:12.894281 15480 consensus_queue.cc:1035] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:12.895939 9908 raft_consensus.cc:1261] T 50f059668c6b4205a15d2c076e2789b0 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:12.895953 11389 raft_consensus.cc:1261] T 50f059668c6b4205a15d2c076e2789b0 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:12.896425 15531 consensus_queue.cc:1035] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:12.896580 15535 consensus_queue.cc:1035] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:12.896828 15013 raft_consensus.cc:2932] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:12.897274 9515 raft_consensus.cc:2932] T 364d227472894712a2ae15f4589a5a5b P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:12.898751 11520 raft_consensus.cc:2932] T 364d227472894712a2ae15f4589a5a5b P 98bef2e94ade4763a66c1ad68259744a [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:12.899338 9369 catalog_manager.cc:5484] T 364d227472894712a2ae15f4589a5a5b P a201db721bda4dc3a4422dc8fa9bf72e reported cstate change: config changed from index 7 to 8, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:12.901044 15481 raft_consensus.cc:2932] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:12.901089 12589 raft_consensus.cc:2932] T 364d227472894712a2ae15f4589a5a5b P 09ec2b380cbf4200af90b0eee891c010 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:12.902890 11389 raft_consensus.cc:2932] T 50f059668c6b4205a15d2c076e2789b0 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:12.903167 12588 raft_consensus.cc:2932] T 50f059668c6b4205a15d2c076e2789b0 P 09ec2b380cbf4200af90b0eee891c010 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:12.904166 9367 catalog_manager.cc:5484] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a reported cstate change: config changed from index 7 to 8, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:12.905968 9908 raft_consensus.cc:2932] T 50f059668c6b4205a15d2c076e2789b0 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:12.909029 9646 consensus_queue.cc:237] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } }
I20240502 07:59:12.909840 9644 consensus_queue.cc:237] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } }
I20240502 07:59:12.910451 11520 raft_consensus.cc:1261] T 364d227472894712a2ae15f4589a5a5b P 98bef2e94ade4763a66c1ad68259744a [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:12.910748 15531 consensus_queue.cc:1035] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:12.911098 11389 raft_consensus.cc:1261] T 50f059668c6b4205a15d2c076e2789b0 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:12.911098 12588 raft_consensus.cc:1261] T 50f059668c6b4205a15d2c076e2789b0 P 09ec2b380cbf4200af90b0eee891c010 [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:12.911338 12589 raft_consensus.cc:1261] T 364d227472894712a2ae15f4589a5a5b P 09ec2b380cbf4200af90b0eee891c010 [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:12.911556 15531 consensus_queue.cc:1035] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:12.912010 15481 consensus_queue.cc:1035] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:12.912024 15531 consensus_queue.cc:1035] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:12.912686 15531 raft_consensus.cc:2932] T 364d227472894712a2ae15f4589a5a5b P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:12.912966 11520 raft_consensus.cc:2932] T 364d227472894712a2ae15f4589a5a5b P 98bef2e94ade4763a66c1ad68259744a [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:12.914116 9358 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 364d227472894712a2ae15f4589a5a5b with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:12.914281 15480 raft_consensus.cc:2932] T 50f059668c6b4205a15d2c076e2789b0 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:12.915042 9367 catalog_manager.cc:5484] T 364d227472894712a2ae15f4589a5a5b P 98bef2e94ade4763a66c1ad68259744a reported cstate change: config changed from index 8 to 9, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:12.915529 9358 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 50f059668c6b4205a15d2c076e2789b0 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:12.916734 12588 raft_consensus.cc:2932] T 50f059668c6b4205a15d2c076e2789b0 P 09ec2b380cbf4200af90b0eee891c010 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:12.918788 12588 raft_consensus.cc:2932] T 364d227472894712a2ae15f4589a5a5b P 09ec2b380cbf4200af90b0eee891c010 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:12.918972 9369 catalog_manager.cc:5484] T 50f059668c6b4205a15d2c076e2789b0 P 09ec2b380cbf4200af90b0eee891c010 reported cstate change: config changed from index 8 to 9, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:12.920192 9493 tablet_service.cc:1514] Processing DeleteTablet for tablet 364d227472894712a2ae15f4589a5a5b with delete_type TABLET_DATA_TOMBSTONED (TS a201db721bda4dc3a4422dc8fa9bf72e not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:41730
I20240502 07:59:12.920761 11389 raft_consensus.cc:2932] T 50f059668c6b4205a15d2c076e2789b0 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:12.921540 9886 tablet_service.cc:1514] Processing DeleteTablet for tablet 50f059668c6b4205a15d2c076e2789b0 with delete_type TABLET_DATA_TOMBSTONED (TS 6cc5b9962ed24441bcba0c07b9f5438e not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:41940
I20240502 07:59:12.922170 15442 tablet_replica.cc:330] T 50f059668c6b4205a15d2c076e2789b0 P 6cc5b9962ed24441bcba0c07b9f5438e: stopping tablet replica
I20240502 07:59:12.922299 15442 raft_consensus.cc:2229] T 50f059668c6b4205a15d2c076e2789b0 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:12.922466 15442 raft_consensus.cc:2258] T 50f059668c6b4205a15d2c076e2789b0 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:12.922641 15552 tablet_replica.cc:330] T 364d227472894712a2ae15f4589a5a5b P a201db721bda4dc3a4422dc8fa9bf72e: stopping tablet replica
I20240502 07:59:12.922793 15552 raft_consensus.cc:2229] T 364d227472894712a2ae15f4589a5a5b P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:12.923082 15552 raft_consensus.cc:2258] T 364d227472894712a2ae15f4589a5a5b P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:12.923192 15442 ts_tablet_manager.cc:1875] T 50f059668c6b4205a15d2c076e2789b0 P 6cc5b9962ed24441bcba0c07b9f5438e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:12.923722 15552 ts_tablet_manager.cc:1875] T 364d227472894712a2ae15f4589a5a5b P a201db721bda4dc3a4422dc8fa9bf72e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:12.925032 15442 ts_tablet_manager.cc:1888] T 50f059668c6b4205a15d2c076e2789b0 P 6cc5b9962ed24441bcba0c07b9f5438e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240502 07:59:12.925087 15442 log.cc:1196] T 50f059668c6b4205a15d2c076e2789b0 P 6cc5b9962ed24441bcba0c07b9f5438e: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal/wals/50f059668c6b4205a15d2c076e2789b0
I20240502 07:59:12.925346 15552 ts_tablet_manager.cc:1888] T 364d227472894712a2ae15f4589a5a5b P a201db721bda4dc3a4422dc8fa9bf72e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240502 07:59:12.925392 15552 log.cc:1196] T 364d227472894712a2ae15f4589a5a5b P a201db721bda4dc3a4422dc8fa9bf72e: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal/wals/364d227472894712a2ae15f4589a5a5b
I20240502 07:59:12.925488 9358 catalog_manager.cc:4830] TS 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379): tablet 50f059668c6b4205a15d2c076e2789b0 (table rebalance_test_table_1 [id=982b41384c8f44b99bbe8eecffb1cd87]) successfully deleted
I20240502 07:59:12.925814 9360 catalog_manager.cc:4830] TS a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551): tablet 364d227472894712a2ae15f4589a5a5b (table rebalance_test_table_6 [id=af6c4a01744d4fcfbdf53cfc8d19500b]) successfully deleted
I20240502 07:59:12.944340 9777 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "92f37161960e4ee894c75d9034434001"
dest_uuid: "1472c64688914149a318c7b8850a3df3"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:49308
I20240502 07:59:12.944449 9777 raft_consensus.cc:595] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Received request to transfer leadership
I20240502 07:59:12.982049 9644 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "12e3bfe638d249c0934feda41b8a8876"
dest_uuid: "8337af1943564c0dad18b45a597b805a"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:38160
I20240502 07:59:12.982173 9644 raft_consensus.cc:595] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Received request to transfer leadership
I20240502 07:59:12.985814 14956 raft_consensus.cc:1050] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e: attempting to promote NON_VOTER 8eb9945f989d4be6b51e1531a1365350 to VOTER
I20240502 07:59:12.986227 14956 consensus_queue.cc:237] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } }
I20240502 07:59:12.987520 11654 raft_consensus.cc:1261] T ee2e79e677f643409e3c8dd49b9517cf P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:12.987608 11389 raft_consensus.cc:1261] T ee2e79e677f643409e3c8dd49b9517cf P 8eb9945f989d4be6b51e1531a1365350 [term 1 LEARNER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:12.987715 9644 raft_consensus.cc:1261] T ee2e79e677f643409e3c8dd49b9517cf P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:12.988063 15356 consensus_queue.cc:1035] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:12.988252 15434 consensus_queue.cc:1035] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:12.988427 14956 consensus_queue.cc:1035] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:12.991417 15434 raft_consensus.cc:2932] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:12.991776 11654 raft_consensus.cc:2932] T ee2e79e677f643409e3c8dd49b9517cf P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:12.992358 9644 raft_consensus.cc:2932] T ee2e79e677f643409e3c8dd49b9517cf P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:12.993944 11389 raft_consensus.cc:2932] T ee2e79e677f643409e3c8dd49b9517cf P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:12.994534 9369 catalog_manager.cc:5484] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e reported cstate change: config changed from index 7 to 8, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:13.006594 9515 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ee2e79e677f643409e3c8dd49b9517cf"
dest_uuid: "a201db721bda4dc3a4422dc8fa9bf72e"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:43580
I20240502 07:59:13.006736 9515 raft_consensus.cc:595] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Received request to transfer leadership
I20240502 07:59:13.018242 9777 consensus_queue.cc:237] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } }
I20240502 07:59:13.019492 11389 raft_consensus.cc:1261] T 70d4db3c3291490db58678522d356688 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:13.019492 9908 raft_consensus.cc:1261] T 70d4db3c3291490db58678522d356688 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240502 07:59:13.019742 9711 consensus_peers.cc:485] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 -> Peer 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73:35761): Couldn't send request to peer 09ec2b380cbf4200af90b0eee891c010. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 70d4db3c3291490db58678522d356688. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:13.019951 15460 consensus_queue.cc:1035] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:13.020310 15460 consensus_queue.cc:1035] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:13.021752 15460 raft_consensus.cc:2932] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } } }
I20240502 07:59:13.023702 11389 raft_consensus.cc:2932] T 70d4db3c3291490db58678522d356688 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } } }
I20240502 07:59:13.024117 9372 catalog_manager.cc:5484] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 reported cstate change: config changed from index 6 to 7, NON_VOTER 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) added. New cstate: current_term: 1 leader_uuid: "1472c64688914149a318c7b8850a3df3" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:13.026072 9908 raft_consensus.cc:2932] T 70d4db3c3291490db58678522d356688 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } } }
I20240502 07:59:13.031244 9777 consensus_queue.cc:237] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } }
I20240502 07:59:13.032382 9515 raft_consensus.cc:1261] T 686b5cf6614d45c18cc776bd79567667 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:13.032495 11654 raft_consensus.cc:1261] T 686b5cf6614d45c18cc776bd79567667 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:13.032742 14872 consensus_queue.cc:1035] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:13.032895 15453 consensus_queue.cc:1035] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20240502 07:59:13.033031 9711 consensus_peers.cc:485] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 -> Peer 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73:35761): Couldn't send request to peer 09ec2b380cbf4200af90b0eee891c010. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 686b5cf6614d45c18cc776bd79567667. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:13.034483 15453 raft_consensus.cc:2932] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } } }
I20240502 07:59:13.034991 11654 raft_consensus.cc:2932] T 686b5cf6614d45c18cc776bd79567667 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } } }
I20240502 07:59:13.035267 9515 raft_consensus.cc:2932] T 686b5cf6614d45c18cc776bd79567667 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } } }
I20240502 07:59:13.037202 9370 catalog_manager.cc:5484] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 reported cstate change: config changed from index 6 to 7, NON_VOTER 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) added. New cstate: current_term: 1 leader_uuid: "1472c64688914149a318c7b8850a3df3" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:13.040522 15461 raft_consensus.cc:1050] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3: attempting to promote NON_VOTER 09ec2b380cbf4200af90b0eee891c010 to VOTER
I20240502 07:59:13.040894 15461 consensus_queue.cc:237] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } }
I20240502 07:59:13.042043 9908 raft_consensus.cc:1261] T 0e4efe560ef74763b80b70665c11d7f1 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:13.042052 12588 raft_consensus.cc:1261] T 0e4efe560ef74763b80b70665c11d7f1 P 09ec2b380cbf4200af90b0eee891c010 [term 1 LEARNER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:13.042492 11520 raft_consensus.cc:1261] T 0e4efe560ef74763b80b70665c11d7f1 P 98bef2e94ade4763a66c1ad68259744a [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:13.042827 14872 consensus_queue.cc:1035] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:13.043006 15460 consensus_queue.cc:1035] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:13.043249 15453 consensus_queue.cc:1035] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:13.045266 15460 raft_consensus.cc:2932] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:13.045723 9908 raft_consensus.cc:2932] T 0e4efe560ef74763b80b70665c11d7f1 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:13.046180 11520 raft_consensus.cc:2932] T 0e4efe560ef74763b80b70665c11d7f1 P 98bef2e94ade4763a66c1ad68259744a [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:13.047585 12588 raft_consensus.cc:2932] T 0e4efe560ef74763b80b70665c11d7f1 P 09ec2b380cbf4200af90b0eee891c010 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:13.047864 9367 catalog_manager.cc:5484] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 reported cstate change: config changed from index 7 to 8, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1472c64688914149a318c7b8850a3df3" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:13.051136 9777 consensus_queue.cc:237] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } }
I20240502 07:59:13.052350 12588 raft_consensus.cc:1261] T 0e4efe560ef74763b80b70665c11d7f1 P 09ec2b380cbf4200af90b0eee891c010 [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:13.052551 11520 raft_consensus.cc:1261] T 0e4efe560ef74763b80b70665c11d7f1 P 98bef2e94ade4763a66c1ad68259744a [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:13.052851 15453 consensus_queue.cc:1035] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:13.053026 14872 consensus_queue.cc:1035] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:13.054582 14872 raft_consensus.cc:2932] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:13.054955 11520 raft_consensus.cc:2932] T 0e4efe560ef74763b80b70665c11d7f1 P 98bef2e94ade4763a66c1ad68259744a [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:13.055481 12588 raft_consensus.cc:2932] T 0e4efe560ef74763b80b70665c11d7f1 P 09ec2b380cbf4200af90b0eee891c010 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:13.056145 9360 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 0e4efe560ef74763b80b70665c11d7f1 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:13.057217 9369 catalog_manager.cc:5484] T 0e4efe560ef74763b80b70665c11d7f1 P 1472c64688914149a318c7b8850a3df3 reported cstate change: config changed from index 8 to 9, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New cstate: current_term: 1 leader_uuid: "1472c64688914149a318c7b8850a3df3" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:13.059948 9886 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e4efe560ef74763b80b70665c11d7f1 with delete_type TABLET_DATA_TOMBSTONED (TS 6cc5b9962ed24441bcba0c07b9f5438e not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:41940
I20240502 07:59:13.060199 15442 tablet_replica.cc:330] T 0e4efe560ef74763b80b70665c11d7f1 P 6cc5b9962ed24441bcba0c07b9f5438e: stopping tablet replica
I20240502 07:59:13.060333 15442 raft_consensus.cc:2229] T 0e4efe560ef74763b80b70665c11d7f1 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:13.060488 15442 raft_consensus.cc:2258] T 0e4efe560ef74763b80b70665c11d7f1 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:13.061084 15442 ts_tablet_manager.cc:1875] T 0e4efe560ef74763b80b70665c11d7f1 P 6cc5b9962ed24441bcba0c07b9f5438e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:13.062762 15442 ts_tablet_manager.cc:1888] T 0e4efe560ef74763b80b70665c11d7f1 P 6cc5b9962ed24441bcba0c07b9f5438e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240502 07:59:13.062824 15442 log.cc:1196] T 0e4efe560ef74763b80b70665c11d7f1 P 6cc5b9962ed24441bcba0c07b9f5438e: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal/wals/0e4efe560ef74763b80b70665c11d7f1
I20240502 07:59:13.063390 9358 catalog_manager.cc:4830] TS 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379): tablet 0e4efe560ef74763b80b70665c11d7f1 (table rebalance_test_table_8 [id=6aecdf32373e4c9db62c5a4844dc38ca]) successfully deleted
I20240502 07:59:13.074568 9777 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "92f37161960e4ee894c75d9034434001"
dest_uuid: "1472c64688914149a318c7b8850a3df3"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:49370
I20240502 07:59:13.074666 9777 raft_consensus.cc:595] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Received request to transfer leadership
I20240502 07:59:13.109041 9644 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "12e3bfe638d249c0934feda41b8a8876"
dest_uuid: "8337af1943564c0dad18b45a597b805a"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:38202
I20240502 07:59:13.109169 9644 raft_consensus.cc:595] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Received request to transfer leadership
I20240502 07:59:13.119024 14091 raft_consensus.cc:1050] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e: attempting to promote NON_VOTER 09ec2b380cbf4200af90b0eee891c010 to VOTER
I20240502 07:59:13.119457 14091 consensus_queue.cc:237] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } }
I20240502 07:59:13.120934 12588 raft_consensus.cc:1261] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 09ec2b380cbf4200af90b0eee891c010 [term 1 LEARNER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:13.121297 9777 raft_consensus.cc:1261] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:13.121620 11259 raft_consensus.cc:1261] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:13.121660 14091 consensus_queue.cc:1035] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:13.121945 15740 consensus_queue.cc:1035] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:13.122328 15740 consensus_queue.cc:1035] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:13.123900 15741 raft_consensus.cc:2932] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:13.124266 9777 raft_consensus.cc:2932] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:13.124541 11259 raft_consensus.cc:2932] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:13.125989 12588 raft_consensus.cc:2932] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 09ec2b380cbf4200af90b0eee891c010 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:13.126507 9372 catalog_manager.cc:5484] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e reported cstate change: config changed from index 7 to 8, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:13.127916 9515 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ee2e79e677f643409e3c8dd49b9517cf"
dest_uuid: "a201db721bda4dc3a4422dc8fa9bf72e"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:43596
I20240502 07:59:13.128031 9515 raft_consensus.cc:595] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Received request to transfer leadership
I20240502 07:59:13.129328 9908 consensus_queue.cc:237] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } }
I20240502 07:59:13.130297 11259 raft_consensus.cc:1261] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:13.130367 12588 raft_consensus.cc:1261] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 09ec2b380cbf4200af90b0eee891c010 [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:13.130796 15741 consensus_queue.cc:1035] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:13.131108 15741 consensus_queue.cc:1035] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:13.132171 15741 raft_consensus.cc:2932] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:13.132519 11259 raft_consensus.cc:2932] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:13.133193 12588 raft_consensus.cc:2932] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 09ec2b380cbf4200af90b0eee891c010 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:13.133821 9358 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 7da9d3b7fe8c40f6a68bb1b7a926d994 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:13.134954 9367 catalog_manager.cc:5484] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 6cc5b9962ed24441bcba0c07b9f5438e reported cstate change: config changed from index 8 to 9, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New cstate: current_term: 1 leader_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:13.135468 14956 raft_consensus.cc:1050] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e: attempting to promote NON_VOTER 98bef2e94ade4763a66c1ad68259744a to VOTER
I20240502 07:59:13.135841 14956 consensus_queue.cc:237] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } }
I20240502 07:59:13.137221 9757 tablet_service.cc:1514] Processing DeleteTablet for tablet 7da9d3b7fe8c40f6a68bb1b7a926d994 with delete_type TABLET_DATA_TOMBSTONED (TS 1472c64688914149a318c7b8850a3df3 not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:57656
I20240502 07:59:13.137347 9777 raft_consensus.cc:1261] T 265eca6fff644d83857ed48a04864a58 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:13.137697 15432 consensus_queue.cc:1035] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:13.138072 11654 raft_consensus.cc:1261] T 265eca6fff644d83857ed48a04864a58 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:13.138799 11520 raft_consensus.cc:1261] T 265eca6fff644d83857ed48a04864a58 P 98bef2e94ade4763a66c1ad68259744a [term 1 LEARNER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:13.139179 15356 consensus_queue.cc:1035] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:13.139349 15432 consensus_queue.cc:1035] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:13.140992 15751 tablet_replica.cc:330] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 1472c64688914149a318c7b8850a3df3: stopping tablet replica
I20240502 07:59:13.141172 15751 raft_consensus.cc:2229] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:13.141117 15356 raft_consensus.cc:2932] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:13.141448 15751 raft_consensus.cc:2258] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:13.142107 15751 ts_tablet_manager.cc:1875] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 1472c64688914149a318c7b8850a3df3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:13.143090 11520 raft_consensus.cc:2932] T 265eca6fff644d83857ed48a04864a58 P 98bef2e94ade4763a66c1ad68259744a [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:13.143158 9777 raft_consensus.cc:2932] T 265eca6fff644d83857ed48a04864a58 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:13.143445 9369 catalog_manager.cc:5484] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e reported cstate change: config changed from index 7 to 8, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:13.144670 11654 raft_consensus.cc:2932] T 265eca6fff644d83857ed48a04864a58 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:13.144824 15751 ts_tablet_manager.cc:1888] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 1472c64688914149a318c7b8850a3df3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240502 07:59:13.144973 15751 log.cc:1196] T 7da9d3b7fe8c40f6a68bb1b7a926d994 P 1472c64688914149a318c7b8850a3df3: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal/wals/7da9d3b7fe8c40f6a68bb1b7a926d994
I20240502 07:59:13.146121 9360 catalog_manager.cc:4830] TS 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191): tablet 7da9d3b7fe8c40f6a68bb1b7a926d994 (table rebalance_test_table_4 [id=a6211a0188464785886f1990ca644211]) successfully deleted
I20240502 07:59:13.148726 9515 consensus_queue.cc:237] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } }
I20240502 07:59:13.149911 11520 raft_consensus.cc:1261] T 265eca6fff644d83857ed48a04864a58 P 98bef2e94ade4763a66c1ad68259744a [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:13.149924 11654 raft_consensus.cc:1261] T 265eca6fff644d83857ed48a04864a58 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:13.150238 15432 consensus_queue.cc:1035] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:13.150367 15434 consensus_queue.cc:1035] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:13.151913 15356 raft_consensus.cc:2932] T 265eca6fff644d83857ed48a04864a58 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:13.152209 11654 raft_consensus.cc:2932] T 265eca6fff644d83857ed48a04864a58 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:13.152456 11520 raft_consensus.cc:2932] T 265eca6fff644d83857ed48a04864a58 P 98bef2e94ade4763a66c1ad68259744a [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:13.153337 9360 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 265eca6fff644d83857ed48a04864a58 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:13.154320 9369 catalog_manager.cc:5484] T 265eca6fff644d83857ed48a04864a58 P a95f1d3af33f4c879fd290ca917d6329 reported cstate change: config changed from index 8 to 9, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:13.156642 9757 tablet_service.cc:1514] Processing DeleteTablet for tablet 265eca6fff644d83857ed48a04864a58 with delete_type TABLET_DATA_TOMBSTONED (TS 1472c64688914149a318c7b8850a3df3 not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:57656
I20240502 07:59:13.156993 15751 tablet_replica.cc:330] T 265eca6fff644d83857ed48a04864a58 P 1472c64688914149a318c7b8850a3df3: stopping tablet replica
I20240502 07:59:13.157115 15751 raft_consensus.cc:2229] T 265eca6fff644d83857ed48a04864a58 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:13.157285 15751 raft_consensus.cc:2258] T 265eca6fff644d83857ed48a04864a58 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:13.157950 15751 ts_tablet_manager.cc:1875] T 265eca6fff644d83857ed48a04864a58 P 1472c64688914149a318c7b8850a3df3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:13.158427 9644 consensus_queue.cc:237] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } }
I20240502 07:59:13.159621 15751 ts_tablet_manager.cc:1888] T 265eca6fff644d83857ed48a04864a58 P 1472c64688914149a318c7b8850a3df3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240502 07:59:13.159686 15751 log.cc:1196] T 265eca6fff644d83857ed48a04864a58 P 1472c64688914149a318c7b8850a3df3: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal/wals/265eca6fff644d83857ed48a04864a58
W20240502 07:59:13.159942 9580 consensus_peers.cc:485] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a -> Peer 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73:35761): Couldn't send request to peer 09ec2b380cbf4200af90b0eee891c010. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f7335ac0209c4e28b6751bb01d34d665. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:13.159960 9515 raft_consensus.cc:1261] T f7335ac0209c4e28b6751bb01d34d665 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:13.159961 11259 raft_consensus.cc:1261] T f7335ac0209c4e28b6751bb01d34d665 P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:13.160270 9360 catalog_manager.cc:4830] TS 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191): tablet 265eca6fff644d83857ed48a04864a58 (table rebalance_test_table_7 [id=7100a319a6ec42499621cff56501ef8f]) successfully deleted
I20240502 07:59:13.160301 15531 consensus_queue.cc:1035] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:13.160439 15535 consensus_queue.cc:1035] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:13.162053 15480 raft_consensus.cc:2932] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } } }
I20240502 07:59:13.163611 9515 raft_consensus.cc:2932] T f7335ac0209c4e28b6751bb01d34d665 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } } }
I20240502 07:59:13.163612 11259 raft_consensus.cc:2932] T f7335ac0209c4e28b6751bb01d34d665 P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } } }
I20240502 07:59:13.165375 9370 catalog_manager.cc:5484] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a reported cstate change: config changed from index 6 to 7, NON_VOTER 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) added. New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:13.170306 15461 raft_consensus.cc:1050] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3: attempting to promote NON_VOTER 09ec2b380cbf4200af90b0eee891c010 to VOTER
I20240502 07:59:13.170675 15461 consensus_queue.cc:237] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } }
I20240502 07:59:13.172276 12588 raft_consensus.cc:1261] T 96dab19eeaea4648913045482b3e5e9e P 09ec2b380cbf4200af90b0eee891c010 [term 1 LEARNER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:13.172601 9644 raft_consensus.cc:1261] T 96dab19eeaea4648913045482b3e5e9e P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:13.172616 9908 raft_consensus.cc:1261] T 96dab19eeaea4648913045482b3e5e9e P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:13.173053 15460 consensus_queue.cc:1035] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:13.173192 14872 consensus_queue.cc:1035] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:13.173502 14872 consensus_queue.cc:1035] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:13.175799 14872 raft_consensus.cc:2932] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:13.176174 9644 raft_consensus.cc:2932] T 96dab19eeaea4648913045482b3e5e9e P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:13.176885 12588 raft_consensus.cc:2932] T 96dab19eeaea4648913045482b3e5e9e P 09ec2b380cbf4200af90b0eee891c010 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:13.178326 9369 catalog_manager.cc:5484] T 96dab19eeaea4648913045482b3e5e9e P 8337af1943564c0dad18b45a597b805a reported cstate change: config changed from index 4 to 5, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1472c64688914149a318c7b8850a3df3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:13.178521 9908 raft_consensus.cc:2932] T 96dab19eeaea4648913045482b3e5e9e P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:13.191517 15535 raft_consensus.cc:1050] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a: attempting to promote NON_VOTER 51e82953ed2247fa9ff24638eab3508a to VOTER
I20240502 07:59:13.191910 15535 consensus_queue.cc:237] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } }
I20240502 07:59:13.193300 9908 raft_consensus.cc:1261] T d562a250cee247a48e573b25ecf00429 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:13.193441 11259 raft_consensus.cc:1261] T d562a250cee247a48e573b25ecf00429 P 51e82953ed2247fa9ff24638eab3508a [term 1 LEARNER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:13.193650 11654 raft_consensus.cc:1261] T d562a250cee247a48e573b25ecf00429 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:13.193769 15532 consensus_queue.cc:1035] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:13.194084 15532 consensus_queue.cc:1035] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:13.194926 15481 consensus_queue.cc:1035] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:13.196233 15480 raft_consensus.cc:2932] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:13.196645 9908 raft_consensus.cc:2932] T d562a250cee247a48e573b25ecf00429 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:13.197357 11654 raft_consensus.cc:2932] T d562a250cee247a48e573b25ecf00429 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:13.197969 11259 raft_consensus.cc:2932] T d562a250cee247a48e573b25ecf00429 P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:13.198524 9777 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "92f37161960e4ee894c75d9034434001"
dest_uuid: "1472c64688914149a318c7b8850a3df3"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:49418
I20240502 07:59:13.198616 9777 raft_consensus.cc:595] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Received request to transfer leadership
I20240502 07:59:13.198940 9369 catalog_manager.cc:5484] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a reported cstate change: config changed from index 7 to 8, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:13.204910 15531 raft_consensus.cc:1050] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a: attempting to promote NON_VOTER 98bef2e94ade4763a66c1ad68259744a to VOTER
I20240502 07:59:13.205271 15531 consensus_queue.cc:237] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } }
I20240502 07:59:13.206651 11259 raft_consensus.cc:1261] T 8ceb129e9cbe4c469e7dfba419719556 P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:13.206693 9777 raft_consensus.cc:1261] T 8ceb129e9cbe4c469e7dfba419719556 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:13.206988 15532 consensus_queue.cc:1035] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:13.207087 11520 raft_consensus.cc:1261] T 8ceb129e9cbe4c469e7dfba419719556 P 98bef2e94ade4763a66c1ad68259744a [term 1 LEARNER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:13.207269 15481 consensus_queue.cc:1035] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:13.207587 15481 consensus_queue.cc:1035] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:13.208992 15481 raft_consensus.cc:2932] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:13.209447 9777 raft_consensus.cc:2932] T 8ceb129e9cbe4c469e7dfba419719556 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:13.210628 11520 raft_consensus.cc:2932] T 8ceb129e9cbe4c469e7dfba419719556 P 98bef2e94ade4763a66c1ad68259744a [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:13.210769 11259 raft_consensus.cc:2932] T 8ceb129e9cbe4c469e7dfba419719556 P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:13.212142 9372 catalog_manager.cc:5484] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a reported cstate change: config changed from index 7 to 8, 98bef2e94ade4763a66c1ad68259744a (127.3.139.71) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:13.214601 9644 consensus_queue.cc:237] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } }
I20240502 07:59:13.215590 11520 raft_consensus.cc:1261] T 8ceb129e9cbe4c469e7dfba419719556 P 98bef2e94ade4763a66c1ad68259744a [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:13.215902 15481 consensus_queue.cc:1035] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:13.216735 15453 raft_consensus.cc:979] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3: : Instructing follower 8eb9945f989d4be6b51e1531a1365350 to start an election
I20240502 07:59:13.216845 15461 raft_consensus.cc:1067] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Signalling peer 8eb9945f989d4be6b51e1531a1365350 to start an election
I20240502 07:59:13.216903 11259 raft_consensus.cc:1261] T 8ceb129e9cbe4c469e7dfba419719556 P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:13.217267 15481 consensus_queue.cc:1035] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:13.217381 11389 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "92f37161960e4ee894c75d9034434001"
dest_uuid: "8eb9945f989d4be6b51e1531a1365350"
from {username='slave'} at 127.3.139.67:59711
I20240502 07:59:13.217559 11389 raft_consensus.cc:482] T 92f37161960e4ee894c75d9034434001 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:13.217624 11389 raft_consensus.cc:3037] T 92f37161960e4ee894c75d9034434001 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:13.217577 15481 raft_consensus.cc:2932] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:13.217913 11520 raft_consensus.cc:2932] T 8ceb129e9cbe4c469e7dfba419719556 P 98bef2e94ade4763a66c1ad68259744a [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:13.218475 11389 raft_consensus.cc:504] T 92f37161960e4ee894c75d9034434001 P 8eb9945f989d4be6b51e1531a1365350 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } }
I20240502 07:59:13.219789 11389 leader_election.cc:288] T 92f37161960e4ee894c75d9034434001 P 8eb9945f989d4be6b51e1531a1365350 [CANDIDATE]: Term 2 election: Requested vote from peers 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191), 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379), 98bef2e94ade4763a66c1ad68259744a (127.3.139.71:46577)
I20240502 07:59:13.219976 9777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92f37161960e4ee894c75d9034434001" candidate_uuid: "8eb9945f989d4be6b51e1531a1365350" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "1472c64688914149a318c7b8850a3df3"
I20240502 07:59:13.220110 9777 raft_consensus.cc:3032] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:59:13.220157 9777 raft_consensus.cc:729] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1472c64688914149a318c7b8850a3df3, State: Running, Role: LEADER
I20240502 07:59:13.220306 9777 consensus_queue.cc:260] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } }
I20240502 07:59:13.220479 9777 raft_consensus.cc:3037] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:13.220782 9908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92f37161960e4ee894c75d9034434001" candidate_uuid: "8eb9945f989d4be6b51e1531a1365350" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "6cc5b9962ed24441bcba0c07b9f5438e"
I20240502 07:59:13.220887 9908 raft_consensus.cc:3037] T 92f37161960e4ee894c75d9034434001 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:13.221278 9777 raft_consensus.cc:2454] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8eb9945f989d4be6b51e1531a1365350 in term 2.
I20240502 07:59:13.221704 9908 raft_consensus.cc:2454] T 92f37161960e4ee894c75d9034434001 P 6cc5b9962ed24441bcba0c07b9f5438e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8eb9945f989d4be6b51e1531a1365350 in term 2.
I20240502 07:59:13.221978 11325 leader_election.cc:302] T 92f37161960e4ee894c75d9034434001 P 8eb9945f989d4be6b51e1531a1365350 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1472c64688914149a318c7b8850a3df3, 6cc5b9962ed24441bcba0c07b9f5438e, 8eb9945f989d4be6b51e1531a1365350; no voters:
I20240502 07:59:13.222177 15268 raft_consensus.cc:2781] T 92f37161960e4ee894c75d9034434001 P 8eb9945f989d4be6b51e1531a1365350 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:13.222234 9358 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 8ceb129e9cbe4c469e7dfba419719556 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:13.222354 15268 raft_consensus.cc:686] T 92f37161960e4ee894c75d9034434001 P 8eb9945f989d4be6b51e1531a1365350 [term 2 LEADER]: Becoming Leader. State: Replica: 8eb9945f989d4be6b51e1531a1365350, State: Running, Role: LEADER
I20240502 07:59:13.222627 15268 consensus_queue.cc:237] T 92f37161960e4ee894c75d9034434001 P 8eb9945f989d4be6b51e1531a1365350 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } }
I20240502 07:59:13.223099 9367 catalog_manager.cc:5484] T 8ceb129e9cbe4c469e7dfba419719556 P 8337af1943564c0dad18b45a597b805a reported cstate change: config changed from index 8 to 9, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:13.224393 11259 raft_consensus.cc:2932] T 8ceb129e9cbe4c469e7dfba419719556 P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:13.225373 9757 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ceb129e9cbe4c469e7dfba419719556 with delete_type TABLET_DATA_TOMBSTONED (TS 1472c64688914149a318c7b8850a3df3 not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:57656
I20240502 07:59:13.225538 15751 tablet_replica.cc:330] T 8ceb129e9cbe4c469e7dfba419719556 P 1472c64688914149a318c7b8850a3df3: stopping tablet replica
I20240502 07:59:13.225668 15751 raft_consensus.cc:2229] T 8ceb129e9cbe4c469e7dfba419719556 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:13.225567 9367 catalog_manager.cc:5484] T 92f37161960e4ee894c75d9034434001 P 8eb9945f989d4be6b51e1531a1365350 reported cstate change: term changed from 1 to 2, leader changed from 1472c64688914149a318c7b8850a3df3 (127.3.139.67) to 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70). New cstate: current_term: 2 leader_uuid: "8eb9945f989d4be6b51e1531a1365350" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:13.225832 15751 raft_consensus.cc:2258] T 8ceb129e9cbe4c469e7dfba419719556 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:13.226707 15751 ts_tablet_manager.cc:1875] T 8ceb129e9cbe4c469e7dfba419719556 P 1472c64688914149a318c7b8850a3df3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:13.228677 15751 ts_tablet_manager.cc:1888] T 8ceb129e9cbe4c469e7dfba419719556 P 1472c64688914149a318c7b8850a3df3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240502 07:59:13.228760 15751 log.cc:1196] T 8ceb129e9cbe4c469e7dfba419719556 P 1472c64688914149a318c7b8850a3df3: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal/wals/8ceb129e9cbe4c469e7dfba419719556
I20240502 07:59:13.229224 9360 catalog_manager.cc:4830] TS 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191): tablet 8ceb129e9cbe4c469e7dfba419719556 (table rebalance_test_table_9 [id=f0ea2346b9b9482e8343d199275d5819]) successfully deleted
I20240502 07:59:13.229830 11520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92f37161960e4ee894c75d9034434001" candidate_uuid: "8eb9945f989d4be6b51e1531a1365350" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "98bef2e94ade4763a66c1ad68259744a"
I20240502 07:59:13.229990 11520 raft_consensus.cc:3037] T 92f37161960e4ee894c75d9034434001 P 98bef2e94ade4763a66c1ad68259744a [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:13.230851 11520 raft_consensus.cc:2454] T 92f37161960e4ee894c75d9034434001 P 98bef2e94ade4763a66c1ad68259744a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8eb9945f989d4be6b51e1531a1365350 in term 2.
I20240502 07:59:13.241711 9644 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "12e3bfe638d249c0934feda41b8a8876"
dest_uuid: "8337af1943564c0dad18b45a597b805a"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:38248
I20240502 07:59:13.241799 9644 raft_consensus.cc:595] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Received request to transfer leadership
I20240502 07:59:13.250744 9644 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d562a250cee247a48e573b25ecf00429"
dest_uuid: "8337af1943564c0dad18b45a597b805a"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:38256
I20240502 07:59:13.250842 9644 raft_consensus.cc:595] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Received request to transfer leadership
I20240502 07:59:13.259147 9515 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ee2e79e677f643409e3c8dd49b9517cf"
dest_uuid: "a201db721bda4dc3a4422dc8fa9bf72e"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:43618
I20240502 07:59:13.259255 9515 raft_consensus.cc:595] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Received request to transfer leadership
I20240502 07:59:13.273730 9777 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "96dab19eeaea4648913045482b3e5e9e"
dest_uuid: "1472c64688914149a318c7b8850a3df3"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:49434
I20240502 07:59:13.273830 9777 raft_consensus.cc:595] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Received request to transfer leadership
I20240502 07:59:13.280567 9908 consensus_queue.cc:237] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } }
I20240502 07:59:13.281728 9777 raft_consensus.cc:1261] T af07c527fff94115a743df1c73c3f4b5 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:13.281745 11389 raft_consensus.cc:1261] T af07c527fff94115a743df1c73c3f4b5 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240502 07:59:13.281970 9844 consensus_peers.cc:485] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e -> Peer 51e82953ed2247fa9ff24638eab3508a (127.3.139.69:45345): Couldn't send request to peer 51e82953ed2247fa9ff24638eab3508a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: af07c527fff94115a743df1c73c3f4b5. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:13.282115 15732 consensus_queue.cc:1035] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:13.282320 15741 consensus_queue.cc:1035] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:13.284011 15733 raft_consensus.cc:2932] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } } }
I20240502 07:59:13.284539 9777 raft_consensus.cc:2932] T af07c527fff94115a743df1c73c3f4b5 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } } }
I20240502 07:59:13.284543 11389 raft_consensus.cc:2932] T af07c527fff94115a743df1c73c3f4b5 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } } }
I20240502 07:59:13.286979 9369 catalog_manager.cc:5484] T af07c527fff94115a743df1c73c3f4b5 P 1472c64688914149a318c7b8850a3df3 reported cstate change: config changed from index 6 to 7, NON_VOTER 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) added. New cstate: current_term: 1 leader_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } } }
I20240502 07:59:13.292773 9515 consensus_queue.cc:237] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } }
I20240502 07:59:13.294014 11389 raft_consensus.cc:1261] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:13.294096 9644 raft_consensus.cc:1261] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:13.294339 15432 consensus_queue.cc:1035] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:13.294670 15432 consensus_queue.cc:1035] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:13.295994 15432 raft_consensus.cc:2932] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } } }
I20240502 07:59:13.296401 11389 raft_consensus.cc:2932] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } } }
W20240502 07:59:13.298310 9449 consensus_peers.cc:485] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e -> Peer 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73:35761): Couldn't send request to peer 09ec2b380cbf4200af90b0eee891c010. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: eddeb6a23ec1418eb35b414f5d52e7b7. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:13.298978 9370 catalog_manager.cc:5484] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8eb9945f989d4be6b51e1531a1365350 reported cstate change: config changed from index 6 to 7, NON_VOTER 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) added. New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } } }
I20240502 07:59:13.299589 9644 raft_consensus.cc:2932] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } } }
I20240502 07:59:13.318794 15481 raft_consensus.cc:979] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a: : Instructing follower a95f1d3af33f4c879fd290ca917d6329 to start an election
I20240502 07:59:13.318889 15481 raft_consensus.cc:1067] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Signalling peer a95f1d3af33f4c879fd290ca917d6329 to start an election
I20240502 07:59:13.319438 11654 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "12e3bfe638d249c0934feda41b8a8876"
dest_uuid: "a95f1d3af33f4c879fd290ca917d6329"
from {username='slave'} at 127.3.139.66:46841
I20240502 07:59:13.319628 11654 raft_consensus.cc:482] T 12e3bfe638d249c0934feda41b8a8876 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:13.319742 11654 raft_consensus.cc:3037] T 12e3bfe638d249c0934feda41b8a8876 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:13.320770 11654 raft_consensus.cc:504] T 12e3bfe638d249c0934feda41b8a8876 P a95f1d3af33f4c879fd290ca917d6329 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } }
I20240502 07:59:13.322098 11654 leader_election.cc:288] T 12e3bfe638d249c0934feda41b8a8876 P a95f1d3af33f4c879fd290ca917d6329 [CANDIDATE]: Term 2 election: Requested vote from peers a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551), 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667), 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70:45329)
I20240502 07:59:13.322790 9644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12e3bfe638d249c0934feda41b8a8876" candidate_uuid: "a95f1d3af33f4c879fd290ca917d6329" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "8337af1943564c0dad18b45a597b805a"
I20240502 07:59:13.322898 9644 raft_consensus.cc:3032] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:59:13.322932 9644 raft_consensus.cc:729] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8337af1943564c0dad18b45a597b805a, State: Running, Role: LEADER
I20240502 07:59:13.323025 9515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12e3bfe638d249c0934feda41b8a8876" candidate_uuid: "a95f1d3af33f4c879fd290ca917d6329" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "a201db721bda4dc3a4422dc8fa9bf72e"
I20240502 07:59:13.323097 9644 consensus_queue.cc:260] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } }
I20240502 07:59:13.323160 9515 raft_consensus.cc:3037] T 12e3bfe638d249c0934feda41b8a8876 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:13.323282 9644 raft_consensus.cc:3037] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:13.324148 9644 raft_consensus.cc:2454] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a95f1d3af33f4c879fd290ca917d6329 in term 2.
I20240502 07:59:13.324184 9515 raft_consensus.cc:2454] T 12e3bfe638d249c0934feda41b8a8876 P a201db721bda4dc3a4422dc8fa9bf72e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a95f1d3af33f4c879fd290ca917d6329 in term 2.
I20240502 07:59:13.324642 11591 leader_election.cc:302] T 12e3bfe638d249c0934feda41b8a8876 P a95f1d3af33f4c879fd290ca917d6329 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8337af1943564c0dad18b45a597b805a, a201db721bda4dc3a4422dc8fa9bf72e, a95f1d3af33f4c879fd290ca917d6329; no voters:
I20240502 07:59:13.325021 15272 raft_consensus.cc:2781] T 12e3bfe638d249c0934feda41b8a8876 P a95f1d3af33f4c879fd290ca917d6329 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:13.325201 15272 raft_consensus.cc:686] T 12e3bfe638d249c0934feda41b8a8876 P a95f1d3af33f4c879fd290ca917d6329 [term 2 LEADER]: Becoming Leader. State: Replica: a95f1d3af33f4c879fd290ca917d6329, State: Running, Role: LEADER
I20240502 07:59:13.325487 15272 consensus_queue.cc:237] T 12e3bfe638d249c0934feda41b8a8876 P a95f1d3af33f4c879fd290ca917d6329 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } }
I20240502 07:59:13.327970 9370 catalog_manager.cc:5484] T 12e3bfe638d249c0934feda41b8a8876 P a95f1d3af33f4c879fd290ca917d6329 reported cstate change: term changed from 1 to 2, leader changed from 8337af1943564c0dad18b45a597b805a (127.3.139.66) to a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72). New cstate: current_term: 2 leader_uuid: "a95f1d3af33f4c879fd290ca917d6329" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:13.331239 11389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12e3bfe638d249c0934feda41b8a8876" candidate_uuid: "a95f1d3af33f4c879fd290ca917d6329" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "8eb9945f989d4be6b51e1531a1365350"
I20240502 07:59:13.331359 11389 raft_consensus.cc:3037] T 12e3bfe638d249c0934feda41b8a8876 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:13.332271 11389 raft_consensus.cc:2454] T 12e3bfe638d249c0934feda41b8a8876 P 8eb9945f989d4be6b51e1531a1365350 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a95f1d3af33f4c879fd290ca917d6329 in term 2.
I20240502 07:59:13.372481 9644 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d562a250cee247a48e573b25ecf00429"
dest_uuid: "8337af1943564c0dad18b45a597b805a"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:38300
I20240502 07:59:13.372606 9644 raft_consensus.cc:595] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Received request to transfer leadership
I20240502 07:59:13.381279 9515 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ee2e79e677f643409e3c8dd49b9517cf"
dest_uuid: "a201db721bda4dc3a4422dc8fa9bf72e"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:43636
I20240502 07:59:13.381387 9515 raft_consensus.cc:595] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Received request to transfer leadership
I20240502 07:59:13.394922 9777 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "96dab19eeaea4648913045482b3e5e9e"
dest_uuid: "1472c64688914149a318c7b8850a3df3"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:49464
I20240502 07:59:13.395028 9777 raft_consensus.cc:595] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Received request to transfer leadership
I20240502 07:59:13.399796 15732 raft_consensus.cc:1050] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e: attempting to promote NON_VOTER 51e82953ed2247fa9ff24638eab3508a to VOTER
I20240502 07:59:13.400175 15732 consensus_queue.cc:237] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } }
I20240502 07:59:13.401512 11259 raft_consensus.cc:1261] T 3255d89d51a94426a78364230c9b249b P 51e82953ed2247fa9ff24638eab3508a [term 1 LEARNER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:13.401510 11654 raft_consensus.cc:1261] T 3255d89d51a94426a78364230c9b249b P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:13.401510 9644 raft_consensus.cc:1261] T 3255d89d51a94426a78364230c9b249b P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:13.401875 15740 consensus_queue.cc:1035] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:13.402211 15741 consensus_queue.cc:1035] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:13.402370 15733 consensus_queue.cc:1035] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:13.404315 15740 raft_consensus.cc:2932] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:13.406105 11654 raft_consensus.cc:2932] T 3255d89d51a94426a78364230c9b249b P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:13.406505 11259 raft_consensus.cc:2932] T 3255d89d51a94426a78364230c9b249b P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:13.406610 9370 catalog_manager.cc:5484] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e reported cstate change: config changed from index 7 to 8, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:13.409579 9908 consensus_queue.cc:237] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } }
I20240502 07:59:13.409979 9644 raft_consensus.cc:2932] T 3255d89d51a94426a78364230c9b249b P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:13.410786 11259 raft_consensus.cc:1261] T 3255d89d51a94426a78364230c9b249b P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:13.411252 11654 raft_consensus.cc:1261] T 3255d89d51a94426a78364230c9b249b P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:13.411674 14091 consensus_queue.cc:1035] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:13.411923 14091 consensus_queue.cc:1035] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:13.413322 15740 raft_consensus.cc:2932] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:13.414502 11654 raft_consensus.cc:2932] T 3255d89d51a94426a78364230c9b249b P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:13.415318 9358 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 3255d89d51a94426a78364230c9b249b with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:13.415365 15986 ts_tablet_manager.cc:902] T 70d4db3c3291490db58678522d356688 P 09ec2b380cbf4200af90b0eee891c010: Initiating tablet copy from peer 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191)
I20240502 07:59:13.415675 9367 catalog_manager.cc:5484] T 3255d89d51a94426a78364230c9b249b P 6cc5b9962ed24441bcba0c07b9f5438e reported cstate change: config changed from index 8 to 9, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New cstate: current_term: 1 leader_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:13.415856 15985 ts_tablet_manager.cc:902] T 686b5cf6614d45c18cc776bd79567667 P 09ec2b380cbf4200af90b0eee891c010: Initiating tablet copy from peer 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191)
I20240502 07:59:13.416747 11259 raft_consensus.cc:2932] T 3255d89d51a94426a78364230c9b249b P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:13.417394 15985 tablet_copy_client.cc:308] T 686b5cf6614d45c18cc776bd79567667 P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.67:43191
I20240502 07:59:13.417667 15986 tablet_copy_client.cc:308] T 70d4db3c3291490db58678522d356688 P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.67:43191
I20240502 07:59:13.418257 9797 tablet_copy_service.cc:140] P 1472c64688914149a318c7b8850a3df3: Received BeginTabletCopySession request for tablet 686b5cf6614d45c18cc776bd79567667 from peer 09ec2b380cbf4200af90b0eee891c010 ({username='slave'} at 127.3.139.73:38883)
I20240502 07:59:13.418273 9796 tablet_copy_service.cc:140] P 1472c64688914149a318c7b8850a3df3: Received BeginTabletCopySession request for tablet 70d4db3c3291490db58678522d356688 from peer 09ec2b380cbf4200af90b0eee891c010 ({username='slave'} at 127.3.139.73:38883)
I20240502 07:59:13.418301 9624 tablet_service.cc:1514] Processing DeleteTablet for tablet 3255d89d51a94426a78364230c9b249b with delete_type TABLET_DATA_TOMBSTONED (TS 8337af1943564c0dad18b45a597b805a not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:53862
I20240502 07:59:13.418362 9797 tablet_copy_service.cc:161] P 1472c64688914149a318c7b8850a3df3: Beginning new tablet copy session on tablet 686b5cf6614d45c18cc776bd79567667 from peer 09ec2b380cbf4200af90b0eee891c010 at {username='slave'} at 127.3.139.73:38883: session id = 09ec2b380cbf4200af90b0eee891c010-686b5cf6614d45c18cc776bd79567667
I20240502 07:59:13.418465 9796 tablet_copy_service.cc:161] P 1472c64688914149a318c7b8850a3df3: Beginning new tablet copy session on tablet 70d4db3c3291490db58678522d356688 from peer 09ec2b380cbf4200af90b0eee891c010 at {username='slave'} at 127.3.139.73:38883: session id = 09ec2b380cbf4200af90b0eee891c010-70d4db3c3291490db58678522d356688
I20240502 07:59:13.418998 15995 tablet_replica.cc:330] T 3255d89d51a94426a78364230c9b249b P 8337af1943564c0dad18b45a597b805a: stopping tablet replica
I20240502 07:59:13.419193 15995 raft_consensus.cc:2229] T 3255d89d51a94426a78364230c9b249b P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:13.419271 9796 tablet_copy_source_session.cc:205] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:13.419479 15995 raft_consensus.cc:2258] T 3255d89d51a94426a78364230c9b249b P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:13.419271 9797 tablet_copy_source_session.cc:205] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:13.419863 15432 raft_consensus.cc:979] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e: : Instructing follower 8eb9945f989d4be6b51e1531a1365350 to start an election
I20240502 07:59:13.419946 15432 raft_consensus.cc:1067] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Signalling peer 8eb9945f989d4be6b51e1531a1365350 to start an election
I20240502 07:59:13.420313 15995 ts_tablet_manager.cc:1875] T 3255d89d51a94426a78364230c9b249b P 8337af1943564c0dad18b45a597b805a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:13.420845 15986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70d4db3c3291490db58678522d356688. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:13.421348 15985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 686b5cf6614d45c18cc776bd79567667. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:13.422384 15995 ts_tablet_manager.cc:1888] T 3255d89d51a94426a78364230c9b249b P 8337af1943564c0dad18b45a597b805a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240502 07:59:13.422456 15995 log.cc:1196] T 3255d89d51a94426a78364230c9b249b P 8337af1943564c0dad18b45a597b805a: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal/wals/3255d89d51a94426a78364230c9b249b
I20240502 07:59:13.422849 15986 tablet_copy_client.cc:783] T 70d4db3c3291490db58678522d356688 P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:13.422871 15985 tablet_copy_client.cc:783] T 686b5cf6614d45c18cc776bd79567667 P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:13.422978 15986 tablet_copy_client.cc:647] T 70d4db3c3291490db58678522d356688 P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:13.423038 15985 tablet_copy_client.cc:647] T 686b5cf6614d45c18cc776bd79567667 P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:13.423079 9358 catalog_manager.cc:4830] TS 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667): tablet 3255d89d51a94426a78364230c9b249b (table rebalance_test_table_2 [id=1b8f60b832d543568624eb22602bb97c]) successfully deleted
I20240502 07:59:13.423144 11391 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ee2e79e677f643409e3c8dd49b9517cf"
dest_uuid: "8eb9945f989d4be6b51e1531a1365350"
from {username='slave'} at 127.3.139.65:52273
I20240502 07:59:13.423269 11391 raft_consensus.cc:482] T ee2e79e677f643409e3c8dd49b9517cf P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:13.423334 11391 raft_consensus.cc:3037] T ee2e79e677f643409e3c8dd49b9517cf P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:13.423987 15994 tablet_copy_client.cc:1002] T 70d4db3c3291490db58678522d356688 P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Injecting latency into file download: 1500
I20240502 07:59:13.424258 11391 raft_consensus.cc:504] T ee2e79e677f643409e3c8dd49b9517cf P 8eb9945f989d4be6b51e1531a1365350 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } }
I20240502 07:59:13.424525 15990 tablet_copy_client.cc:1002] T 686b5cf6614d45c18cc776bd79567667 P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Injecting latency into file download: 1500
I20240502 07:59:13.424822 11391 leader_election.cc:288] T ee2e79e677f643409e3c8dd49b9517cf P 8eb9945f989d4be6b51e1531a1365350 [CANDIDATE]: Term 2 election: Requested vote from peers a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551), 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667), a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72:36271)
I20240502 07:59:13.425081 9644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee2e79e677f643409e3c8dd49b9517cf" candidate_uuid: "8eb9945f989d4be6b51e1531a1365350" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "8337af1943564c0dad18b45a597b805a"
I20240502 07:59:13.425174 9644 raft_consensus.cc:3037] T ee2e79e677f643409e3c8dd49b9517cf P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:13.425289 9515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee2e79e677f643409e3c8dd49b9517cf" candidate_uuid: "8eb9945f989d4be6b51e1531a1365350" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "a201db721bda4dc3a4422dc8fa9bf72e"
I20240502 07:59:13.425396 9515 raft_consensus.cc:3032] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:59:13.425451 9515 raft_consensus.cc:729] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a201db721bda4dc3a4422dc8fa9bf72e, State: Running, Role: LEADER
I20240502 07:59:13.425601 9515 consensus_queue.cc:260] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } }
I20240502 07:59:13.425823 9515 raft_consensus.cc:3037] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:13.425962 9644 raft_consensus.cc:2454] T ee2e79e677f643409e3c8dd49b9517cf P 8337af1943564c0dad18b45a597b805a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8eb9945f989d4be6b51e1531a1365350 in term 2.
I20240502 07:59:13.426733 9515 raft_consensus.cc:2454] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8eb9945f989d4be6b51e1531a1365350 in term 2.
I20240502 07:59:13.427953 11327 leader_election.cc:302] T ee2e79e677f643409e3c8dd49b9517cf P 8eb9945f989d4be6b51e1531a1365350 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8337af1943564c0dad18b45a597b805a, 8eb9945f989d4be6b51e1531a1365350, a201db721bda4dc3a4422dc8fa9bf72e; no voters:
I20240502 07:59:13.428107 15268 raft_consensus.cc:2781] T ee2e79e677f643409e3c8dd49b9517cf P 8eb9945f989d4be6b51e1531a1365350 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:13.428210 15268 raft_consensus.cc:686] T ee2e79e677f643409e3c8dd49b9517cf P 8eb9945f989d4be6b51e1531a1365350 [term 2 LEADER]: Becoming Leader. State: Replica: 8eb9945f989d4be6b51e1531a1365350, State: Running, Role: LEADER
I20240502 07:59:13.428356 15268 consensus_queue.cc:237] T ee2e79e677f643409e3c8dd49b9517cf P 8eb9945f989d4be6b51e1531a1365350 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } }
I20240502 07:59:13.429855 9369 catalog_manager.cc:5484] T ee2e79e677f643409e3c8dd49b9517cf P 8eb9945f989d4be6b51e1531a1365350 reported cstate change: term changed from 1 to 2, leader changed from a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) to 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70). New cstate: current_term: 2 leader_uuid: "8eb9945f989d4be6b51e1531a1365350" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:13.432797 15461 raft_consensus.cc:1050] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3: attempting to promote NON_VOTER 8eb9945f989d4be6b51e1531a1365350 to VOTER
I20240502 07:59:13.433104 15461 consensus_queue.cc:237] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } }
I20240502 07:59:13.433128 11654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee2e79e677f643409e3c8dd49b9517cf" candidate_uuid: "8eb9945f989d4be6b51e1531a1365350" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "a95f1d3af33f4c879fd290ca917d6329"
I20240502 07:59:13.433287 11654 raft_consensus.cc:3037] T ee2e79e677f643409e3c8dd49b9517cf P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:13.434055 11654 raft_consensus.cc:2454] T ee2e79e677f643409e3c8dd49b9517cf P a95f1d3af33f4c879fd290ca917d6329 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8eb9945f989d4be6b51e1531a1365350 in term 2.
I20240502 07:59:13.434314 11655 raft_consensus.cc:1261] T d37b8a7f016647a7a03baedc37ed5419 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:13.434370 9515 raft_consensus.cc:1261] T d37b8a7f016647a7a03baedc37ed5419 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:13.434579 11391 raft_consensus.cc:1261] T d37b8a7f016647a7a03baedc37ed5419 P 8eb9945f989d4be6b51e1531a1365350 [term 1 LEARNER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:13.434970 15453 consensus_queue.cc:1035] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:13.435438 15460 consensus_queue.cc:1035] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:13.437245 15460 consensus_queue.cc:1035] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:13.438060 15460 raft_consensus.cc:2932] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:13.439534 11654 raft_consensus.cc:2932] T d37b8a7f016647a7a03baedc37ed5419 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:13.440109 9369 catalog_manager.cc:5484] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 reported cstate change: config changed from index 7 to 8, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1472c64688914149a318c7b8850a3df3" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:13.442159 9515 raft_consensus.cc:2932] T d37b8a7f016647a7a03baedc37ed5419 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:13.443086 11391 raft_consensus.cc:2932] T d37b8a7f016647a7a03baedc37ed5419 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:13.447263 9777 consensus_queue.cc:237] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } }
I20240502 07:59:13.448311 11391 raft_consensus.cc:1261] T d37b8a7f016647a7a03baedc37ed5419 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:13.448365 11654 raft_consensus.cc:1261] T d37b8a7f016647a7a03baedc37ed5419 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:13.448659 15453 consensus_queue.cc:1035] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:13.448900 14872 consensus_queue.cc:1035] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:13.449935 15453 raft_consensus.cc:2932] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:13.450361 11391 raft_consensus.cc:2932] T d37b8a7f016647a7a03baedc37ed5419 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:13.451191 9360 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet d37b8a7f016647a7a03baedc37ed5419 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:13.451948 11654 raft_consensus.cc:2932] T d37b8a7f016647a7a03baedc37ed5419 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:13.452263 9370 catalog_manager.cc:5484] T d37b8a7f016647a7a03baedc37ed5419 P 1472c64688914149a318c7b8850a3df3 reported cstate change: config changed from index 8 to 9, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New cstate: current_term: 1 leader_uuid: "1472c64688914149a318c7b8850a3df3" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:13.454694 9493 tablet_service.cc:1514] Processing DeleteTablet for tablet d37b8a7f016647a7a03baedc37ed5419 with delete_type TABLET_DATA_TOMBSTONED (TS a201db721bda4dc3a4422dc8fa9bf72e not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:41730
I20240502 07:59:13.455008 16017 tablet_replica.cc:330] T d37b8a7f016647a7a03baedc37ed5419 P a201db721bda4dc3a4422dc8fa9bf72e: stopping tablet replica
I20240502 07:59:13.455288 16017 raft_consensus.cc:2229] T d37b8a7f016647a7a03baedc37ed5419 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:13.455514 16017 raft_consensus.cc:2258] T d37b8a7f016647a7a03baedc37ed5419 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:13.456187 16017 ts_tablet_manager.cc:1875] T d37b8a7f016647a7a03baedc37ed5419 P a201db721bda4dc3a4422dc8fa9bf72e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:13.458220 16017 ts_tablet_manager.cc:1888] T d37b8a7f016647a7a03baedc37ed5419 P a201db721bda4dc3a4422dc8fa9bf72e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240502 07:59:13.458297 16017 log.cc:1196] T d37b8a7f016647a7a03baedc37ed5419 P a201db721bda4dc3a4422dc8fa9bf72e: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal/wals/d37b8a7f016647a7a03baedc37ed5419
I20240502 07:59:13.458714 9360 catalog_manager.cc:4830] TS a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551): tablet d37b8a7f016647a7a03baedc37ed5419 (table rebalance_test_table_3 [id=47c27ad65ee1405898b667c2642e1b3a]) successfully deleted
I20240502 07:59:13.477628 9644 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d562a250cee247a48e573b25ecf00429"
dest_uuid: "8337af1943564c0dad18b45a597b805a"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:38342
I20240502 07:59:13.477730 9644 raft_consensus.cc:595] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Received request to transfer leadership
I20240502 07:59:13.495716 9777 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "96dab19eeaea4648913045482b3e5e9e"
dest_uuid: "1472c64688914149a318c7b8850a3df3"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:49504
I20240502 07:59:13.495832 9777 raft_consensus.cc:595] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Received request to transfer leadership
I20240502 07:59:13.533565 9644 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d562a250cee247a48e573b25ecf00429"
dest_uuid: "8337af1943564c0dad18b45a597b805a"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:38372
I20240502 07:59:13.533679 9644 raft_consensus.cc:595] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Received request to transfer leadership
I20240502 07:59:13.551748 9777 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "96dab19eeaea4648913045482b3e5e9e"
dest_uuid: "1472c64688914149a318c7b8850a3df3"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:49532
I20240502 07:59:13.551853 9777 raft_consensus.cc:595] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Received request to transfer leadership
I20240502 07:59:13.576510 15460 raft_consensus.cc:979] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3: : Instructing follower 6cc5b9962ed24441bcba0c07b9f5438e to start an election
I20240502 07:59:13.576599 15460 raft_consensus.cc:1067] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Signalling peer 6cc5b9962ed24441bcba0c07b9f5438e to start an election
I20240502 07:59:13.576973 9908 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "96dab19eeaea4648913045482b3e5e9e"
dest_uuid: "6cc5b9962ed24441bcba0c07b9f5438e"
from {username='slave'} at 127.3.139.67:38273
I20240502 07:59:13.577086 9908 raft_consensus.cc:482] T 96dab19eeaea4648913045482b3e5e9e P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:13.577140 9908 raft_consensus.cc:3037] T 96dab19eeaea4648913045482b3e5e9e P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:13.577888 9908 raft_consensus.cc:504] T 96dab19eeaea4648913045482b3e5e9e P 6cc5b9962ed24441bcba0c07b9f5438e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } }
I20240502 07:59:13.578246 9908 leader_election.cc:288] T 96dab19eeaea4648913045482b3e5e9e P 6cc5b9962ed24441bcba0c07b9f5438e [CANDIDATE]: Term 2 election: Requested vote from peers 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191), 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667), 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73:35761)
I20240502 07:59:13.578636 9644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96dab19eeaea4648913045482b3e5e9e" candidate_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "8337af1943564c0dad18b45a597b805a"
I20240502 07:59:13.578755 9644 raft_consensus.cc:3037] T 96dab19eeaea4648913045482b3e5e9e P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:13.578836 9777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96dab19eeaea4648913045482b3e5e9e" candidate_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "1472c64688914149a318c7b8850a3df3"
I20240502 07:59:13.578827 12588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96dab19eeaea4648913045482b3e5e9e" candidate_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "09ec2b380cbf4200af90b0eee891c010"
I20240502 07:59:13.578944 9777 raft_consensus.cc:3032] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:59:13.578984 9777 raft_consensus.cc:729] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1472c64688914149a318c7b8850a3df3, State: Running, Role: LEADER
I20240502 07:59:13.579001 12588 raft_consensus.cc:3037] T 96dab19eeaea4648913045482b3e5e9e P 09ec2b380cbf4200af90b0eee891c010 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:13.579140 9777 consensus_queue.cc:260] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } }
I20240502 07:59:13.579330 9777 raft_consensus.cc:3037] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:13.579728 9644 raft_consensus.cc:2454] T 96dab19eeaea4648913045482b3e5e9e P 8337af1943564c0dad18b45a597b805a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cc5b9962ed24441bcba0c07b9f5438e in term 2.
I20240502 07:59:13.580050 12588 raft_consensus.cc:2454] T 96dab19eeaea4648913045482b3e5e9e P 09ec2b380cbf4200af90b0eee891c010 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cc5b9962ed24441bcba0c07b9f5438e in term 2.
I20240502 07:59:13.580261 9777 raft_consensus.cc:2454] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cc5b9962ed24441bcba0c07b9f5438e in term 2.
I20240502 07:59:13.580395 9842 leader_election.cc:302] T 96dab19eeaea4648913045482b3e5e9e P 6cc5b9962ed24441bcba0c07b9f5438e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 09ec2b380cbf4200af90b0eee891c010, 6cc5b9962ed24441bcba0c07b9f5438e, 8337af1943564c0dad18b45a597b805a; no voters:
I20240502 07:59:13.580668 15733 raft_consensus.cc:2781] T 96dab19eeaea4648913045482b3e5e9e P 6cc5b9962ed24441bcba0c07b9f5438e [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:13.580750 15733 raft_consensus.cc:686] T 96dab19eeaea4648913045482b3e5e9e P 6cc5b9962ed24441bcba0c07b9f5438e [term 2 LEADER]: Becoming Leader. State: Replica: 6cc5b9962ed24441bcba0c07b9f5438e, State: Running, Role: LEADER
I20240502 07:59:13.580902 15733 consensus_queue.cc:237] T 96dab19eeaea4648913045482b3e5e9e P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } }
I20240502 07:59:13.582736 9369 catalog_manager.cc:5484] T 96dab19eeaea4648913045482b3e5e9e P 6cc5b9962ed24441bcba0c07b9f5438e reported cstate change: term changed from 1 to 2, leader changed from 1472c64688914149a318c7b8850a3df3 (127.3.139.67) to 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68). New cstate: current_term: 2 leader_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:13.598569 15480 raft_consensus.cc:979] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a: : Instructing follower 6cc5b9962ed24441bcba0c07b9f5438e to start an election
I20240502 07:59:13.598671 15480 raft_consensus.cc:1067] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Signalling peer 6cc5b9962ed24441bcba0c07b9f5438e to start an election
I20240502 07:59:13.599109 9908 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d562a250cee247a48e573b25ecf00429"
dest_uuid: "6cc5b9962ed24441bcba0c07b9f5438e"
from {username='slave'} at 127.3.139.66:58991
I20240502 07:59:13.599229 9908 raft_consensus.cc:482] T d562a250cee247a48e573b25ecf00429 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:13.599288 9908 raft_consensus.cc:3037] T d562a250cee247a48e573b25ecf00429 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:13.599988 9908 raft_consensus.cc:504] T d562a250cee247a48e573b25ecf00429 P 6cc5b9962ed24441bcba0c07b9f5438e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } }
I20240502 07:59:13.600389 9908 leader_election.cc:288] T d562a250cee247a48e573b25ecf00429 P 6cc5b9962ed24441bcba0c07b9f5438e [CANDIDATE]: Term 2 election: Requested vote from peers 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667), a95f1d3af33f4c879fd290ca917d6329 (127.3.139.72:36271), 51e82953ed2247fa9ff24638eab3508a (127.3.139.69:45345)
I20240502 07:59:13.600699 11654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d562a250cee247a48e573b25ecf00429" candidate_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "a95f1d3af33f4c879fd290ca917d6329"
I20240502 07:59:13.600811 11654 raft_consensus.cc:3037] T d562a250cee247a48e573b25ecf00429 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:13.600797 9644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d562a250cee247a48e573b25ecf00429" candidate_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "8337af1943564c0dad18b45a597b805a"
I20240502 07:59:13.600895 9644 raft_consensus.cc:3032] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:59:13.600930 9644 raft_consensus.cc:729] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8337af1943564c0dad18b45a597b805a, State: Running, Role: LEADER
I20240502 07:59:13.600884 11259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d562a250cee247a48e573b25ecf00429" candidate_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "51e82953ed2247fa9ff24638eab3508a"
I20240502 07:59:13.601061 11259 raft_consensus.cc:3037] T d562a250cee247a48e573b25ecf00429 P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:13.601052 9644 consensus_queue.cc:260] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } }
I20240502 07:59:13.601220 9644 raft_consensus.cc:3037] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:13.601687 11654 raft_consensus.cc:2454] T d562a250cee247a48e573b25ecf00429 P a95f1d3af33f4c879fd290ca917d6329 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cc5b9962ed24441bcba0c07b9f5438e in term 2.
I20240502 07:59:13.602043 9644 raft_consensus.cc:2454] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cc5b9962ed24441bcba0c07b9f5438e in term 2.
I20240502 07:59:13.602062 11259 raft_consensus.cc:2454] T d562a250cee247a48e573b25ecf00429 P 51e82953ed2247fa9ff24638eab3508a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cc5b9962ed24441bcba0c07b9f5438e in term 2.
I20240502 07:59:13.602360 9844 leader_election.cc:302] T d562a250cee247a48e573b25ecf00429 P 6cc5b9962ed24441bcba0c07b9f5438e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 51e82953ed2247fa9ff24638eab3508a, 6cc5b9962ed24441bcba0c07b9f5438e, a95f1d3af33f4c879fd290ca917d6329; no voters:
I20240502 07:59:13.602509 15733 raft_consensus.cc:2781] T d562a250cee247a48e573b25ecf00429 P 6cc5b9962ed24441bcba0c07b9f5438e [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:13.602593 15733 raft_consensus.cc:686] T d562a250cee247a48e573b25ecf00429 P 6cc5b9962ed24441bcba0c07b9f5438e [term 2 LEADER]: Becoming Leader. State: Replica: 6cc5b9962ed24441bcba0c07b9f5438e, State: Running, Role: LEADER
I20240502 07:59:13.602742 15733 consensus_queue.cc:237] T d562a250cee247a48e573b25ecf00429 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } }
I20240502 07:59:13.604434 9369 catalog_manager.cc:5484] T d562a250cee247a48e573b25ecf00429 P 6cc5b9962ed24441bcba0c07b9f5438e reported cstate change: term changed from 1 to 2, leader changed from 8337af1943564c0dad18b45a597b805a (127.3.139.66) to 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68). New cstate: current_term: 2 leader_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:13.624727 16121 ts_tablet_manager.cc:902] T f7335ac0209c4e28b6751bb01d34d665 P 09ec2b380cbf4200af90b0eee891c010: Initiating tablet copy from peer 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667)
I20240502 07:59:13.625144 16121 tablet_copy_client.cc:308] T f7335ac0209c4e28b6751bb01d34d665 P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.66:46667
I20240502 07:59:13.625630 9666 tablet_copy_service.cc:140] P 8337af1943564c0dad18b45a597b805a: Received BeginTabletCopySession request for tablet f7335ac0209c4e28b6751bb01d34d665 from peer 09ec2b380cbf4200af90b0eee891c010 ({username='slave'} at 127.3.139.73:58997)
I20240502 07:59:13.625715 9666 tablet_copy_service.cc:161] P 8337af1943564c0dad18b45a597b805a: Beginning new tablet copy session on tablet f7335ac0209c4e28b6751bb01d34d665 from peer 09ec2b380cbf4200af90b0eee891c010 at {username='slave'} at 127.3.139.73:58997: session id = 09ec2b380cbf4200af90b0eee891c010-f7335ac0209c4e28b6751bb01d34d665
I20240502 07:59:13.626310 9666 tablet_copy_source_session.cc:205] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:13.627121 16121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7335ac0209c4e28b6751bb01d34d665. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:13.628688 16121 tablet_copy_client.cc:783] T f7335ac0209c4e28b6751bb01d34d665 P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:13.628816 16121 tablet_copy_client.cc:647] T f7335ac0209c4e28b6751bb01d34d665 P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:13.629773 16125 tablet_copy_client.cc:1002] T f7335ac0209c4e28b6751bb01d34d665 P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Injecting latency into file download: 1500
I20240502 07:59:13.657526 9777 raft_consensus.cc:1261] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [term 2 FOLLOWER]: Refusing update from remote peer 8eb9945f989d4be6b51e1531a1365350: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240502 07:59:13.657959 15268 consensus_queue.cc:1035] T 92f37161960e4ee894c75d9034434001 P 8eb9945f989d4be6b51e1531a1365350 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:13.683993 9908 raft_consensus.cc:1261] T 92f37161960e4ee894c75d9034434001 P 6cc5b9962ed24441bcba0c07b9f5438e [term 2 FOLLOWER]: Refusing update from remote peer 8eb9945f989d4be6b51e1531a1365350: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240502 07:59:13.684465 15268 consensus_queue.cc:1035] T 92f37161960e4ee894c75d9034434001 P 8eb9945f989d4be6b51e1531a1365350 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:13.687759 11520 raft_consensus.cc:1261] T 92f37161960e4ee894c75d9034434001 P 98bef2e94ade4763a66c1ad68259744a [term 2 FOLLOWER]: Refusing update from remote peer 8eb9945f989d4be6b51e1531a1365350: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240502 07:59:13.688534 16127 consensus_queue.cc:1035] T 92f37161960e4ee894c75d9034434001 P 8eb9945f989d4be6b51e1531a1365350 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:13.773627 16151 ts_tablet_manager.cc:902] T eddeb6a23ec1418eb35b414f5d52e7b7 P 09ec2b380cbf4200af90b0eee891c010: Initiating tablet copy from peer a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551)
I20240502 07:59:13.774101 16151 tablet_copy_client.cc:308] T eddeb6a23ec1418eb35b414f5d52e7b7 P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.65:39551
I20240502 07:59:13.777421 9534 tablet_copy_service.cc:140] P a201db721bda4dc3a4422dc8fa9bf72e: Received BeginTabletCopySession request for tablet eddeb6a23ec1418eb35b414f5d52e7b7 from peer 09ec2b380cbf4200af90b0eee891c010 ({username='slave'} at 127.3.139.73:56219)
I20240502 07:59:13.777519 9534 tablet_copy_service.cc:161] P a201db721bda4dc3a4422dc8fa9bf72e: Beginning new tablet copy session on tablet eddeb6a23ec1418eb35b414f5d52e7b7 from peer 09ec2b380cbf4200af90b0eee891c010 at {username='slave'} at 127.3.139.73:56219: session id = 09ec2b380cbf4200af90b0eee891c010-eddeb6a23ec1418eb35b414f5d52e7b7
I20240502 07:59:13.778363 9534 tablet_copy_source_session.cc:205] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:13.779557 16151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eddeb6a23ec1418eb35b414f5d52e7b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:13.781504 16151 tablet_copy_client.cc:783] T eddeb6a23ec1418eb35b414f5d52e7b7 P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:13.781673 16151 tablet_copy_client.cc:647] T eddeb6a23ec1418eb35b414f5d52e7b7 P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:13.782711 16155 tablet_copy_client.cc:1002] T eddeb6a23ec1418eb35b414f5d52e7b7 P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Injecting latency into file download: 1500
I20240502 07:59:13.788829 9644 raft_consensus.cc:1261] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [term 2 FOLLOWER]: Refusing update from remote peer a95f1d3af33f4c879fd290ca917d6329: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240502 07:59:13.789292 15908 consensus_queue.cc:1035] T 12e3bfe638d249c0934feda41b8a8876 P a95f1d3af33f4c879fd290ca917d6329 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:13.806190 11391 raft_consensus.cc:1261] T 12e3bfe638d249c0934feda41b8a8876 P 8eb9945f989d4be6b51e1531a1365350 [term 2 FOLLOWER]: Refusing update from remote peer a95f1d3af33f4c879fd290ca917d6329: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240502 07:59:13.806582 15908 consensus_queue.cc:1035] T 12e3bfe638d249c0934feda41b8a8876 P a95f1d3af33f4c879fd290ca917d6329 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:13.810182 9515 raft_consensus.cc:1261] T 12e3bfe638d249c0934feda41b8a8876 P a201db721bda4dc3a4422dc8fa9bf72e [term 2 FOLLOWER]: Refusing update from remote peer a95f1d3af33f4c879fd290ca917d6329: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240502 07:59:13.810564 15908 consensus_queue.cc:1035] T 12e3bfe638d249c0934feda41b8a8876 P a95f1d3af33f4c879fd290ca917d6329 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:13.835296 11654 raft_consensus.cc:1261] T ee2e79e677f643409e3c8dd49b9517cf P a95f1d3af33f4c879fd290ca917d6329 [term 2 FOLLOWER]: Refusing update from remote peer 8eb9945f989d4be6b51e1531a1365350: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240502 07:59:13.835676 16127 consensus_queue.cc:1035] T ee2e79e677f643409e3c8dd49b9517cf P 8eb9945f989d4be6b51e1531a1365350 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:13.879192 16191 ts_tablet_manager.cc:902] T af07c527fff94115a743df1c73c3f4b5 P 51e82953ed2247fa9ff24638eab3508a: Initiating tablet copy from peer 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379)
I20240502 07:59:13.879684 16191 tablet_copy_client.cc:308] T af07c527fff94115a743df1c73c3f4b5 P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.68:37379
I20240502 07:59:13.880203 9928 tablet_copy_service.cc:140] P 6cc5b9962ed24441bcba0c07b9f5438e: Received BeginTabletCopySession request for tablet af07c527fff94115a743df1c73c3f4b5 from peer 51e82953ed2247fa9ff24638eab3508a ({username='slave'} at 127.3.139.69:48381)
I20240502 07:59:13.880307 9928 tablet_copy_service.cc:161] P 6cc5b9962ed24441bcba0c07b9f5438e: Beginning new tablet copy session on tablet af07c527fff94115a743df1c73c3f4b5 from peer 51e82953ed2247fa9ff24638eab3508a at {username='slave'} at 127.3.139.69:48381: session id = 51e82953ed2247fa9ff24638eab3508a-af07c527fff94115a743df1c73c3f4b5
I20240502 07:59:13.881031 9928 tablet_copy_source_session.cc:205] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:13.881948 16191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af07c527fff94115a743df1c73c3f4b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:13.883963 16191 tablet_copy_client.cc:783] T af07c527fff94115a743df1c73c3f4b5 P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:13.884116 16191 tablet_copy_client.cc:647] T af07c527fff94115a743df1c73c3f4b5 P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:13.885267 16195 tablet_copy_client.cc:1002] T af07c527fff94115a743df1c73c3f4b5 P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Injecting latency into file download: 1500
I20240502 07:59:13.902952 9515 raft_consensus.cc:1261] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [term 2 FOLLOWER]: Refusing update from remote peer 8eb9945f989d4be6b51e1531a1365350: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240502 07:59:13.903421 16127 consensus_queue.cc:1035] T ee2e79e677f643409e3c8dd49b9517cf P 8eb9945f989d4be6b51e1531a1365350 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:13.905766 9644 raft_consensus.cc:1261] T ee2e79e677f643409e3c8dd49b9517cf P 8337af1943564c0dad18b45a597b805a [term 2 FOLLOWER]: Refusing update from remote peer 8eb9945f989d4be6b51e1531a1365350: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240502 07:59:13.906180 15817 consensus_queue.cc:1035] T ee2e79e677f643409e3c8dd49b9517cf P 8eb9945f989d4be6b51e1531a1365350 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:14.019599 11654 raft_consensus.cc:1261] T d562a250cee247a48e573b25ecf00429 P a95f1d3af33f4c879fd290ca917d6329 [term 2 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240502 07:59:14.020061 14091 consensus_queue.cc:1035] T d562a250cee247a48e573b25ecf00429 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:14.025506 9777 raft_consensus.cc:1261] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3 [term 2 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 07:59:14.025898 14091 consensus_queue.cc:1035] T 96dab19eeaea4648913045482b3e5e9e P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:14.087369 11259 raft_consensus.cc:1261] T d562a250cee247a48e573b25ecf00429 P 51e82953ed2247fa9ff24638eab3508a [term 2 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240502 07:59:14.087813 14091 consensus_queue.cc:1035] T d562a250cee247a48e573b25ecf00429 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:14.090759 9644 raft_consensus.cc:1261] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [term 2 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240502 07:59:14.091163 15733 consensus_queue.cc:1035] T d562a250cee247a48e573b25ecf00429 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:14.092181 9908 consensus_queue.cc:237] T d562a250cee247a48e573b25ecf00429 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 8, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } }
I20240502 07:59:14.093817 11259 raft_consensus.cc:1261] T d562a250cee247a48e573b25ecf00429 P 51e82953ed2247fa9ff24638eab3508a [term 2 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240502 07:59:14.093830 11654 raft_consensus.cc:1261] T d562a250cee247a48e573b25ecf00429 P a95f1d3af33f4c879fd290ca917d6329 [term 2 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240502 07:59:14.094115 12588 raft_consensus.cc:1261] T 96dab19eeaea4648913045482b3e5e9e P 09ec2b380cbf4200af90b0eee891c010 [term 2 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 07:59:14.094476 14091 consensus_queue.cc:1035] T d562a250cee247a48e573b25ecf00429 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:14.094560 15733 consensus_queue.cc:1035] T 96dab19eeaea4648913045482b3e5e9e P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:14.094720 14091 consensus_queue.cc:1035] T d562a250cee247a48e573b25ecf00429 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:14.096760 15733 raft_consensus.cc:2932] T d562a250cee247a48e573b25ecf00429 P 6cc5b9962ed24441bcba0c07b9f5438e [term 2 LEADER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:14.096938 11259 raft_consensus.cc:2932] T d562a250cee247a48e573b25ecf00429 P 51e82953ed2247fa9ff24638eab3508a [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:14.098991 9358 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet d562a250cee247a48e573b25ecf00429 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:14.099326 9644 raft_consensus.cc:1261] T 96dab19eeaea4648913045482b3e5e9e P 8337af1943564c0dad18b45a597b805a [term 2 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 07:59:14.100013 11654 raft_consensus.cc:2932] T d562a250cee247a48e573b25ecf00429 P a95f1d3af33f4c879fd290ca917d6329 [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:14.100167 9370 catalog_manager.cc:5484] T d562a250cee247a48e573b25ecf00429 P 51e82953ed2247fa9ff24638eab3508a reported cstate change: config changed from index 8 to 10, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New cstate: current_term: 2 leader_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:14.100481 16257 consensus_queue.cc:1035] T 96dab19eeaea4648913045482b3e5e9e P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:14.103021 9624 tablet_service.cc:1514] Processing DeleteTablet for tablet d562a250cee247a48e573b25ecf00429 with delete_type TABLET_DATA_TOMBSTONED (TS 8337af1943564c0dad18b45a597b805a not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:53862
I20240502 07:59:14.103353 16259 tablet_replica.cc:330] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a: stopping tablet replica
I20240502 07:59:14.103484 16259 raft_consensus.cc:2229] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:14.103595 16259 raft_consensus.cc:2258] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:14.104213 16259 ts_tablet_manager.cc:1875] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:14.106477 16259 ts_tablet_manager.cc:1888] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.9
I20240502 07:59:14.106588 16259 log.cc:1196] T d562a250cee247a48e573b25ecf00429 P 8337af1943564c0dad18b45a597b805a: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal/wals/d562a250cee247a48e573b25ecf00429
I20240502 07:59:14.107111 9358 catalog_manager.cc:4830] TS 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667): tablet d562a250cee247a48e573b25ecf00429 (table rebalance_test_table_1 [id=982b41384c8f44b99bbe8eecffb1cd87]) successfully deleted
I20240502 07:59:14.120347 11391 consensus_queue.cc:237] T 92f37161960e4ee894c75d9034434001 P 8eb9945f989d4be6b51e1531a1365350 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 8, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } }
I20240502 07:59:14.121410 9908 raft_consensus.cc:1261] T 92f37161960e4ee894c75d9034434001 P 6cc5b9962ed24441bcba0c07b9f5438e [term 2 FOLLOWER]: Refusing update from remote peer 8eb9945f989d4be6b51e1531a1365350: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240502 07:59:14.121412 11520 raft_consensus.cc:1261] T 92f37161960e4ee894c75d9034434001 P 98bef2e94ade4763a66c1ad68259744a [term 2 FOLLOWER]: Refusing update from remote peer 8eb9945f989d4be6b51e1531a1365350: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240502 07:59:14.121760 15268 consensus_queue.cc:1035] T 92f37161960e4ee894c75d9034434001 P 8eb9945f989d4be6b51e1531a1365350 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:14.121980 15817 consensus_queue.cc:1035] T 92f37161960e4ee894c75d9034434001 P 8eb9945f989d4be6b51e1531a1365350 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:14.123785 16127 raft_consensus.cc:2932] T 92f37161960e4ee894c75d9034434001 P 8eb9945f989d4be6b51e1531a1365350 [term 2 LEADER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:14.124212 11520 raft_consensus.cc:2932] T 92f37161960e4ee894c75d9034434001 P 98bef2e94ade4763a66c1ad68259744a [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:14.124222 9908 raft_consensus.cc:2932] T 92f37161960e4ee894c75d9034434001 P 6cc5b9962ed24441bcba0c07b9f5438e [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:14.125830 9358 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 92f37161960e4ee894c75d9034434001 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:14.127147 9372 catalog_manager.cc:5484] T 92f37161960e4ee894c75d9034434001 P 6cc5b9962ed24441bcba0c07b9f5438e reported cstate change: config changed from index 8 to 10, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New cstate: current_term: 2 leader_uuid: "8eb9945f989d4be6b51e1531a1365350" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } }
I20240502 07:59:14.129400 9757 tablet_service.cc:1514] Processing DeleteTablet for tablet 92f37161960e4ee894c75d9034434001 with delete_type TABLET_DATA_TOMBSTONED (TS 1472c64688914149a318c7b8850a3df3 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:57656
I20240502 07:59:14.129741 16261 tablet_replica.cc:330] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3: stopping tablet replica
I20240502 07:59:14.129884 16261 raft_consensus.cc:2229] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:14.129997 16261 raft_consensus.cc:2258] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:14.130524 16261 ts_tablet_manager.cc:1875] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:14.132267 16261 ts_tablet_manager.cc:1888] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.9
I20240502 07:59:14.132330 16261 log.cc:1196] T 92f37161960e4ee894c75d9034434001 P 1472c64688914149a318c7b8850a3df3: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal/wals/92f37161960e4ee894c75d9034434001
I20240502 07:59:14.132776 9360 catalog_manager.cc:4830] TS 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191): tablet 92f37161960e4ee894c75d9034434001 (table rebalance_test_table_0 [id=efdfc231272e43a5afefacf0ec75ec57]) successfully deleted
I20240502 07:59:14.222429 11654 consensus_queue.cc:237] T 12e3bfe638d249c0934feda41b8a8876 P a95f1d3af33f4c879fd290ca917d6329 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 8, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } }
I20240502 07:59:14.223593 9515 raft_consensus.cc:1261] T 12e3bfe638d249c0934feda41b8a8876 P a201db721bda4dc3a4422dc8fa9bf72e [term 2 FOLLOWER]: Refusing update from remote peer a95f1d3af33f4c879fd290ca917d6329: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240502 07:59:14.223600 11391 raft_consensus.cc:1261] T 12e3bfe638d249c0934feda41b8a8876 P 8eb9945f989d4be6b51e1531a1365350 [term 2 FOLLOWER]: Refusing update from remote peer a95f1d3af33f4c879fd290ca917d6329: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240502 07:59:14.223999 15272 consensus_queue.cc:1035] T 12e3bfe638d249c0934feda41b8a8876 P a95f1d3af33f4c879fd290ca917d6329 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:14.224258 16183 consensus_queue.cc:1035] T 12e3bfe638d249c0934feda41b8a8876 P a95f1d3af33f4c879fd290ca917d6329 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:14.225991 15908 raft_consensus.cc:2932] T 12e3bfe638d249c0934feda41b8a8876 P a95f1d3af33f4c879fd290ca917d6329 [term 2 LEADER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:14.226532 11391 raft_consensus.cc:2932] T 12e3bfe638d249c0934feda41b8a8876 P 8eb9945f989d4be6b51e1531a1365350 [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:14.226533 9515 raft_consensus.cc:2932] T 12e3bfe638d249c0934feda41b8a8876 P a201db721bda4dc3a4422dc8fa9bf72e [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } }
I20240502 07:59:14.227622 9359 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 12e3bfe638d249c0934feda41b8a8876 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:14.228889 9367 catalog_manager.cc:5484] T 12e3bfe638d249c0934feda41b8a8876 P a95f1d3af33f4c879fd290ca917d6329 reported cstate change: config changed from index 8 to 10, VOTER 8337af1943564c0dad18b45a597b805a (127.3.139.66) evicted. New cstate: current_term: 2 leader_uuid: "a95f1d3af33f4c879fd290ca917d6329" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:14.231171 9624 tablet_service.cc:1514] Processing DeleteTablet for tablet 12e3bfe638d249c0934feda41b8a8876 with delete_type TABLET_DATA_TOMBSTONED (TS 8337af1943564c0dad18b45a597b805a not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:53862
I20240502 07:59:14.231730 16259 tablet_replica.cc:330] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a: stopping tablet replica
I20240502 07:59:14.231860 16259 raft_consensus.cc:2229] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:14.231966 16259 raft_consensus.cc:2258] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:14.232574 16259 ts_tablet_manager.cc:1875] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:14.234143 16259 ts_tablet_manager.cc:1888] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.9
I20240502 07:59:14.234205 16259 log.cc:1196] T 12e3bfe638d249c0934feda41b8a8876 P 8337af1943564c0dad18b45a597b805a: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal/wals/12e3bfe638d249c0934feda41b8a8876
I20240502 07:59:14.234611 9358 catalog_manager.cc:4830] TS 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667): tablet 12e3bfe638d249c0934feda41b8a8876 (table rebalance_test_table_5 [id=51b1431ec18d4bb8b4d5ae1909ae1322]) successfully deleted
I20240502 07:59:14.395821 11391 consensus_queue.cc:237] T ee2e79e677f643409e3c8dd49b9517cf P 8eb9945f989d4be6b51e1531a1365350 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 8, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } }
I20240502 07:59:14.396977 9644 raft_consensus.cc:1261] T ee2e79e677f643409e3c8dd49b9517cf P 8337af1943564c0dad18b45a597b805a [term 2 FOLLOWER]: Refusing update from remote peer 8eb9945f989d4be6b51e1531a1365350: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240502 07:59:14.396994 11654 raft_consensus.cc:1261] T ee2e79e677f643409e3c8dd49b9517cf P a95f1d3af33f4c879fd290ca917d6329 [term 2 FOLLOWER]: Refusing update from remote peer 8eb9945f989d4be6b51e1531a1365350: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240502 07:59:14.397341 15268 consensus_queue.cc:1035] T ee2e79e677f643409e3c8dd49b9517cf P 8eb9945f989d4be6b51e1531a1365350 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:14.397544 15817 consensus_queue.cc:1035] T ee2e79e677f643409e3c8dd49b9517cf P 8eb9945f989d4be6b51e1531a1365350 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:14.399026 15268 raft_consensus.cc:2932] T ee2e79e677f643409e3c8dd49b9517cf P 8eb9945f989d4be6b51e1531a1365350 [term 2 LEADER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:14.399637 9644 raft_consensus.cc:2932] T ee2e79e677f643409e3c8dd49b9517cf P 8337af1943564c0dad18b45a597b805a [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:14.399637 11654 raft_consensus.cc:2932] T ee2e79e677f643409e3c8dd49b9517cf P a95f1d3af33f4c879fd290ca917d6329 [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:14.400468 9358 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet ee2e79e677f643409e3c8dd49b9517cf with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:14.401985 9372 catalog_manager.cc:5484] T ee2e79e677f643409e3c8dd49b9517cf P 8337af1943564c0dad18b45a597b805a reported cstate change: config changed from index 8 to 10, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New cstate: current_term: 2 leader_uuid: "8eb9945f989d4be6b51e1531a1365350" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } }
I20240502 07:59:14.404564 9493 tablet_service.cc:1514] Processing DeleteTablet for tablet ee2e79e677f643409e3c8dd49b9517cf with delete_type TABLET_DATA_TOMBSTONED (TS a201db721bda4dc3a4422dc8fa9bf72e not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:41730
I20240502 07:59:14.404911 16348 tablet_replica.cc:330] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e: stopping tablet replica
I20240502 07:59:14.405025 16348 raft_consensus.cc:2229] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:14.405112 16348 raft_consensus.cc:2258] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:14.405665 16348 ts_tablet_manager.cc:1875] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:14.407450 16348 ts_tablet_manager.cc:1888] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.9
I20240502 07:59:14.407514 16348 log.cc:1196] T ee2e79e677f643409e3c8dd49b9517cf P a201db721bda4dc3a4422dc8fa9bf72e: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal/wals/ee2e79e677f643409e3c8dd49b9517cf
I20240502 07:59:14.407984 9360 catalog_manager.cc:4830] TS a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551): tablet ee2e79e677f643409e3c8dd49b9517cf (table rebalance_test_table_4 [id=a6211a0188464785886f1990ca644211]) successfully deleted
I20240502 07:59:14.652246 9908 consensus_queue.cc:237] T 96dab19eeaea4648913045482b3e5e9e P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } }
I20240502 07:59:14.653326 9644 raft_consensus.cc:1261] T 96dab19eeaea4648913045482b3e5e9e P 8337af1943564c0dad18b45a597b805a [term 2 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240502 07:59:14.653411 12588 raft_consensus.cc:1261] T 96dab19eeaea4648913045482b3e5e9e P 09ec2b380cbf4200af90b0eee891c010 [term 2 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240502 07:59:14.653697 16254 consensus_queue.cc:1035] T 96dab19eeaea4648913045482b3e5e9e P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:14.653906 16252 consensus_queue.cc:1035] T 96dab19eeaea4648913045482b3e5e9e P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:14.655407 16257 raft_consensus.cc:2932] T 96dab19eeaea4648913045482b3e5e9e P 6cc5b9962ed24441bcba0c07b9f5438e [term 2 LEADER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:14.656666 9358 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 96dab19eeaea4648913045482b3e5e9e with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:14.657193 12588 raft_consensus.cc:2932] T 96dab19eeaea4648913045482b3e5e9e P 09ec2b380cbf4200af90b0eee891c010 [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:14.657474 9367 catalog_manager.cc:5484] T 96dab19eeaea4648913045482b3e5e9e P 6cc5b9962ed24441bcba0c07b9f5438e reported cstate change: config changed from index 5 to 7, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New cstate: current_term: 2 leader_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:14.658607 9644 raft_consensus.cc:2932] T 96dab19eeaea4648913045482b3e5e9e P 8337af1943564c0dad18b45a597b805a [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:14.659901 9757 tablet_service.cc:1514] Processing DeleteTablet for tablet 96dab19eeaea4648913045482b3e5e9e with delete_type TABLET_DATA_TOMBSTONED (TS 1472c64688914149a318c7b8850a3df3 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:57656
I20240502 07:59:14.660316 16415 tablet_replica.cc:330] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3: stopping tablet replica
I20240502 07:59:14.660487 16415 raft_consensus.cc:2229] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:14.660616 16415 raft_consensus.cc:2258] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:14.661233 16415 ts_tablet_manager.cc:1875] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:14.663102 16415 ts_tablet_manager.cc:1888] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.6
I20240502 07:59:14.663163 16415 log.cc:1196] T 96dab19eeaea4648913045482b3e5e9e P 1472c64688914149a318c7b8850a3df3: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal/wals/96dab19eeaea4648913045482b3e5e9e
I20240502 07:59:14.663651 9360 catalog_manager.cc:4830] TS 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191): tablet 96dab19eeaea4648913045482b3e5e9e (table rebalance_test_table_3 [id=47c27ad65ee1405898b667c2642e1b3a]) successfully deleted
I20240502 07:59:14.877408 9644 consensus_queue.cc:237] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } }
I20240502 07:59:14.878722 11521 raft_consensus.cc:1261] T ecd0995c31fe4e708712c62ca60511c9 P 98bef2e94ade4763a66c1ad68259744a [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:14.878851 9777 raft_consensus.cc:1261] T ecd0995c31fe4e708712c62ca60511c9 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:14.879112 16221 consensus_queue.cc:1035] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:14.879292 15480 consensus_queue.cc:1035] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20240502 07:59:14.879482 9580 consensus_peers.cc:485] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a -> Peer 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73:35761): Couldn't send request to peer 09ec2b380cbf4200af90b0eee891c010. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ecd0995c31fe4e708712c62ca60511c9. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:14.881268 15480 raft_consensus.cc:2932] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } } }
I20240502 07:59:14.881654 11521 raft_consensus.cc:2932] T ecd0995c31fe4e708712c62ca60511c9 P 98bef2e94ade4763a66c1ad68259744a [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } } }
I20240502 07:59:14.882787 9777 raft_consensus.cc:2932] T ecd0995c31fe4e708712c62ca60511c9 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } } }
I20240502 07:59:14.884783 9369 catalog_manager.cc:5484] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a reported cstate change: config changed from index 6 to 7, NON_VOTER 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) added. New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:14.924293 15986 tablet_copy_client.cc:523] T 70d4db3c3291490db58678522d356688 P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:14.924963 15985 tablet_copy_client.cc:523] T 686b5cf6614d45c18cc776bd79567667 P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:14.925802 15986 tablet_bootstrap.cc:492] T 70d4db3c3291490db58678522d356688 P 09ec2b380cbf4200af90b0eee891c010: Bootstrap starting.
I20240502 07:59:14.926309 15985 tablet_bootstrap.cc:492] T 686b5cf6614d45c18cc776bd79567667 P 09ec2b380cbf4200af90b0eee891c010: Bootstrap starting.
I20240502 07:59:14.943025 15986 tablet_bootstrap.cc:492] T 70d4db3c3291490db58678522d356688 P 09ec2b380cbf4200af90b0eee891c010: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:14.943315 15986 tablet_bootstrap.cc:492] T 70d4db3c3291490db58678522d356688 P 09ec2b380cbf4200af90b0eee891c010: Bootstrap complete.
I20240502 07:59:14.943462 15986 ts_tablet_manager.cc:1367] T 70d4db3c3291490db58678522d356688 P 09ec2b380cbf4200af90b0eee891c010: Time spent bootstrapping tablet: real 0.018s user 0.010s sys 0.009s
I20240502 07:59:14.943863 15985 tablet_bootstrap.cc:492] T 686b5cf6614d45c18cc776bd79567667 P 09ec2b380cbf4200af90b0eee891c010: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:14.943826 15986 raft_consensus.cc:348] T 70d4db3c3291490db58678522d356688 P 09ec2b380cbf4200af90b0eee891c010 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } }
I20240502 07:59:14.943938 15986 raft_consensus.cc:729] T 70d4db3c3291490db58678522d356688 P 09ec2b380cbf4200af90b0eee891c010 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 09ec2b380cbf4200af90b0eee891c010, State: Initialized, Role: LEARNER
I20240502 07:59:14.944047 15986 consensus_queue.cc:260] T 70d4db3c3291490db58678522d356688 P 09ec2b380cbf4200af90b0eee891c010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } }
I20240502 07:59:14.944118 15985 tablet_bootstrap.cc:492] T 686b5cf6614d45c18cc776bd79567667 P 09ec2b380cbf4200af90b0eee891c010: Bootstrap complete.
I20240502 07:59:14.944255 15985 ts_tablet_manager.cc:1367] T 686b5cf6614d45c18cc776bd79567667 P 09ec2b380cbf4200af90b0eee891c010: Time spent bootstrapping tablet: real 0.018s user 0.013s sys 0.005s
I20240502 07:59:14.944293 15986 ts_tablet_manager.cc:1398] T 70d4db3c3291490db58678522d356688 P 09ec2b380cbf4200af90b0eee891c010: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:14.944624 15985 raft_consensus.cc:348] T 686b5cf6614d45c18cc776bd79567667 P 09ec2b380cbf4200af90b0eee891c010 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } }
I20240502 07:59:14.944826 9797 tablet_copy_service.cc:342] P 1472c64688914149a318c7b8850a3df3: Request end of tablet copy session 09ec2b380cbf4200af90b0eee891c010-70d4db3c3291490db58678522d356688 received from {username='slave'} at 127.3.139.73:38883
I20240502 07:59:14.944866 15985 raft_consensus.cc:729] T 686b5cf6614d45c18cc776bd79567667 P 09ec2b380cbf4200af90b0eee891c010 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 09ec2b380cbf4200af90b0eee891c010, State: Initialized, Role: LEARNER
I20240502 07:59:14.944895 9797 tablet_copy_service.cc:433] P 1472c64688914149a318c7b8850a3df3: ending tablet copy session 09ec2b380cbf4200af90b0eee891c010-70d4db3c3291490db58678522d356688 on tablet 70d4db3c3291490db58678522d356688 with peer 09ec2b380cbf4200af90b0eee891c010
I20240502 07:59:14.944970 15985 consensus_queue.cc:260] T 686b5cf6614d45c18cc776bd79567667 P 09ec2b380cbf4200af90b0eee891c010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } }
I20240502 07:59:14.945233 15985 ts_tablet_manager.cc:1398] T 686b5cf6614d45c18cc776bd79567667 P 09ec2b380cbf4200af90b0eee891c010: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240502 07:59:14.945844 9797 tablet_copy_service.cc:342] P 1472c64688914149a318c7b8850a3df3: Request end of tablet copy session 09ec2b380cbf4200af90b0eee891c010-686b5cf6614d45c18cc776bd79567667 received from {username='slave'} at 127.3.139.73:38883
I20240502 07:59:14.945933 9797 tablet_copy_service.cc:433] P 1472c64688914149a318c7b8850a3df3: ending tablet copy session 09ec2b380cbf4200af90b0eee891c010-686b5cf6614d45c18cc776bd79567667 on tablet 686b5cf6614d45c18cc776bd79567667 with peer 09ec2b380cbf4200af90b0eee891c010
I20240502 07:59:15.130079 16121 tablet_copy_client.cc:523] T f7335ac0209c4e28b6751bb01d34d665 P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:15.131659 16121 tablet_bootstrap.cc:492] T f7335ac0209c4e28b6751bb01d34d665 P 09ec2b380cbf4200af90b0eee891c010: Bootstrap starting.
I20240502 07:59:15.153776 16121 tablet_bootstrap.cc:492] T f7335ac0209c4e28b6751bb01d34d665 P 09ec2b380cbf4200af90b0eee891c010: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:15.154057 16121 tablet_bootstrap.cc:492] T f7335ac0209c4e28b6751bb01d34d665 P 09ec2b380cbf4200af90b0eee891c010: Bootstrap complete.
I20240502 07:59:15.154304 16121 ts_tablet_manager.cc:1367] T f7335ac0209c4e28b6751bb01d34d665 P 09ec2b380cbf4200af90b0eee891c010: Time spent bootstrapping tablet: real 0.023s user 0.019s sys 0.003s
I20240502 07:59:15.154934 16121 raft_consensus.cc:348] T f7335ac0209c4e28b6751bb01d34d665 P 09ec2b380cbf4200af90b0eee891c010 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } }
I20240502 07:59:15.155191 16121 raft_consensus.cc:729] T f7335ac0209c4e28b6751bb01d34d665 P 09ec2b380cbf4200af90b0eee891c010 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 09ec2b380cbf4200af90b0eee891c010, State: Initialized, Role: LEARNER
I20240502 07:59:15.155360 16121 consensus_queue.cc:260] T f7335ac0209c4e28b6751bb01d34d665 P 09ec2b380cbf4200af90b0eee891c010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } }
I20240502 07:59:15.155755 16121 ts_tablet_manager.cc:1398] T f7335ac0209c4e28b6751bb01d34d665 P 09ec2b380cbf4200af90b0eee891c010: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:15.156744 9666 tablet_copy_service.cc:342] P 8337af1943564c0dad18b45a597b805a: Request end of tablet copy session 09ec2b380cbf4200af90b0eee891c010-f7335ac0209c4e28b6751bb01d34d665 received from {username='slave'} at 127.3.139.73:58997
I20240502 07:59:15.156828 9666 tablet_copy_service.cc:433] P 8337af1943564c0dad18b45a597b805a: ending tablet copy session 09ec2b380cbf4200af90b0eee891c010-f7335ac0209c4e28b6751bb01d34d665 on tablet f7335ac0209c4e28b6751bb01d34d665 with peer 09ec2b380cbf4200af90b0eee891c010
I20240502 07:59:15.255419 12588 raft_consensus.cc:1203] T 70d4db3c3291490db58678522d356688 P 09ec2b380cbf4200af90b0eee891c010 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240502 07:59:15.282976 16151 tablet_copy_client.cc:523] T eddeb6a23ec1418eb35b414f5d52e7b7 P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:15.284224 16151 tablet_bootstrap.cc:492] T eddeb6a23ec1418eb35b414f5d52e7b7 P 09ec2b380cbf4200af90b0eee891c010: Bootstrap starting.
I20240502 07:59:15.302327 16151 tablet_bootstrap.cc:492] T eddeb6a23ec1418eb35b414f5d52e7b7 P 09ec2b380cbf4200af90b0eee891c010: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:15.302531 16151 tablet_bootstrap.cc:492] T eddeb6a23ec1418eb35b414f5d52e7b7 P 09ec2b380cbf4200af90b0eee891c010: Bootstrap complete.
I20240502 07:59:15.302657 16151 ts_tablet_manager.cc:1367] T eddeb6a23ec1418eb35b414f5d52e7b7 P 09ec2b380cbf4200af90b0eee891c010: Time spent bootstrapping tablet: real 0.018s user 0.012s sys 0.004s
I20240502 07:59:15.302979 16151 raft_consensus.cc:348] T eddeb6a23ec1418eb35b414f5d52e7b7 P 09ec2b380cbf4200af90b0eee891c010 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } }
I20240502 07:59:15.303118 16151 raft_consensus.cc:729] T eddeb6a23ec1418eb35b414f5d52e7b7 P 09ec2b380cbf4200af90b0eee891c010 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 09ec2b380cbf4200af90b0eee891c010, State: Initialized, Role: LEARNER
I20240502 07:59:15.303223 16151 consensus_queue.cc:260] T eddeb6a23ec1418eb35b414f5d52e7b7 P 09ec2b380cbf4200af90b0eee891c010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } }
I20240502 07:59:15.303489 16151 ts_tablet_manager.cc:1398] T eddeb6a23ec1418eb35b414f5d52e7b7 P 09ec2b380cbf4200af90b0eee891c010: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240502 07:59:15.303963 9534 tablet_copy_service.cc:342] P a201db721bda4dc3a4422dc8fa9bf72e: Request end of tablet copy session 09ec2b380cbf4200af90b0eee891c010-eddeb6a23ec1418eb35b414f5d52e7b7 received from {username='slave'} at 127.3.139.73:56219
I20240502 07:59:15.304042 9534 tablet_copy_service.cc:433] P a201db721bda4dc3a4422dc8fa9bf72e: ending tablet copy session 09ec2b380cbf4200af90b0eee891c010-eddeb6a23ec1418eb35b414f5d52e7b7 on tablet eddeb6a23ec1418eb35b414f5d52e7b7 with peer 09ec2b380cbf4200af90b0eee891c010
I20240502 07:59:15.352878 12588 raft_consensus.cc:1203] T 686b5cf6614d45c18cc776bd79567667 P 09ec2b380cbf4200af90b0eee891c010 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240502 07:59:15.380182 16151 ts_tablet_manager.cc:902] T ecd0995c31fe4e708712c62ca60511c9 P 09ec2b380cbf4200af90b0eee891c010: Initiating tablet copy from peer 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667)
I20240502 07:59:15.380609 16151 tablet_copy_client.cc:308] T ecd0995c31fe4e708712c62ca60511c9 P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.66:46667
I20240502 07:59:15.381052 9666 tablet_copy_service.cc:140] P 8337af1943564c0dad18b45a597b805a: Received BeginTabletCopySession request for tablet ecd0995c31fe4e708712c62ca60511c9 from peer 09ec2b380cbf4200af90b0eee891c010 ({username='slave'} at 127.3.139.73:58997)
I20240502 07:59:15.381145 9666 tablet_copy_service.cc:161] P 8337af1943564c0dad18b45a597b805a: Beginning new tablet copy session on tablet ecd0995c31fe4e708712c62ca60511c9 from peer 09ec2b380cbf4200af90b0eee891c010 at {username='slave'} at 127.3.139.73:58997: session id = 09ec2b380cbf4200af90b0eee891c010-ecd0995c31fe4e708712c62ca60511c9
I20240502 07:59:15.381790 9666 tablet_copy_source_session.cc:205] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:15.382589 16151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecd0995c31fe4e708712c62ca60511c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:15.384634 16151 tablet_copy_client.cc:783] T ecd0995c31fe4e708712c62ca60511c9 P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:15.384778 16151 tablet_copy_client.cc:647] T ecd0995c31fe4e708712c62ca60511c9 P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:15.385566 16191 tablet_copy_client.cc:523] T af07c527fff94115a743df1c73c3f4b5 P 51e82953ed2247fa9ff24638eab3508a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:15.385735 16541 tablet_copy_client.cc:1002] T ecd0995c31fe4e708712c62ca60511c9 P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Injecting latency into file download: 1500
I20240502 07:59:15.386646 16191 tablet_bootstrap.cc:492] T af07c527fff94115a743df1c73c3f4b5 P 51e82953ed2247fa9ff24638eab3508a: Bootstrap starting.
I20240502 07:59:15.406018 16191 tablet_bootstrap.cc:492] T af07c527fff94115a743df1c73c3f4b5 P 51e82953ed2247fa9ff24638eab3508a: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:15.406270 16191 tablet_bootstrap.cc:492] T af07c527fff94115a743df1c73c3f4b5 P 51e82953ed2247fa9ff24638eab3508a: Bootstrap complete.
I20240502 07:59:15.406430 16191 ts_tablet_manager.cc:1367] T af07c527fff94115a743df1c73c3f4b5 P 51e82953ed2247fa9ff24638eab3508a: Time spent bootstrapping tablet: real 0.020s user 0.016s sys 0.001s
I20240502 07:59:15.406786 16191 raft_consensus.cc:348] T af07c527fff94115a743df1c73c3f4b5 P 51e82953ed2247fa9ff24638eab3508a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } }
I20240502 07:59:15.406900 16191 raft_consensus.cc:729] T af07c527fff94115a743df1c73c3f4b5 P 51e82953ed2247fa9ff24638eab3508a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 51e82953ed2247fa9ff24638eab3508a, State: Initialized, Role: LEARNER
I20240502 07:59:15.406991 16191 consensus_queue.cc:260] T af07c527fff94115a743df1c73c3f4b5 P 51e82953ed2247fa9ff24638eab3508a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: true } }
I20240502 07:59:15.407294 16191 ts_tablet_manager.cc:1398] T af07c527fff94115a743df1c73c3f4b5 P 51e82953ed2247fa9ff24638eab3508a: Time spent starting tablet: real 0.001s user 0.001s sys 0.003s
I20240502 07:59:15.407830 9928 tablet_copy_service.cc:342] P 6cc5b9962ed24441bcba0c07b9f5438e: Request end of tablet copy session 51e82953ed2247fa9ff24638eab3508a-af07c527fff94115a743df1c73c3f4b5 received from {username='slave'} at 127.3.139.69:48381
I20240502 07:59:15.407917 9928 tablet_copy_service.cc:433] P 6cc5b9962ed24441bcba0c07b9f5438e: ending tablet copy session 51e82953ed2247fa9ff24638eab3508a-af07c527fff94115a743df1c73c3f4b5 on tablet af07c527fff94115a743df1c73c3f4b5 with peer 51e82953ed2247fa9ff24638eab3508a
I20240502 07:59:15.409482 12588 raft_consensus.cc:1203] T f7335ac0209c4e28b6751bb01d34d665 P 09ec2b380cbf4200af90b0eee891c010 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240502 07:59:15.534572 12588 raft_consensus.cc:1203] T eddeb6a23ec1418eb35b414f5d52e7b7 P 09ec2b380cbf4200af90b0eee891c010 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240502 07:59:15.721947 16587 raft_consensus.cc:1050] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3: attempting to promote NON_VOTER 09ec2b380cbf4200af90b0eee891c010 to VOTER
I20240502 07:59:15.722404 16587 consensus_queue.cc:237] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } }
I20240502 07:59:15.723780 12588 raft_consensus.cc:1261] T 70d4db3c3291490db58678522d356688 P 09ec2b380cbf4200af90b0eee891c010 [term 1 LEARNER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:15.724073 11391 raft_consensus.cc:1261] T 70d4db3c3291490db58678522d356688 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:15.724145 9908 raft_consensus.cc:1261] T 70d4db3c3291490db58678522d356688 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:15.724479 16587 consensus_queue.cc:1035] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:15.724697 14872 consensus_queue.cc:1035] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:15.724885 16589 consensus_queue.cc:1035] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:15.730075 16589 raft_consensus.cc:2932] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:15.732206 11391 raft_consensus.cc:2932] T 70d4db3c3291490db58678522d356688 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:15.732889 9367 catalog_manager.cc:5484] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 reported cstate change: config changed from index 7 to 8, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1472c64688914149a318c7b8850a3df3" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:15.734371 12588 raft_consensus.cc:2932] T 70d4db3c3291490db58678522d356688 P 09ec2b380cbf4200af90b0eee891c010 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:15.735358 9908 raft_consensus.cc:2932] T 70d4db3c3291490db58678522d356688 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } attrs { replace: true } } peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:15.739387 9777 consensus_queue.cc:237] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } }
I20240502 07:59:15.740347 11391 raft_consensus.cc:1261] T 70d4db3c3291490db58678522d356688 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:15.740468 12588 raft_consensus.cc:1261] T 70d4db3c3291490db58678522d356688 P 09ec2b380cbf4200af90b0eee891c010 [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:15.740676 16593 consensus_queue.cc:1035] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:15.740892 16589 consensus_queue.cc:1035] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:15.742375 14872 raft_consensus.cc:2932] T 70d4db3c3291490db58678522d356688 P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:15.743134 11391 raft_consensus.cc:2932] T 70d4db3c3291490db58678522d356688 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:15.744390 9360 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 70d4db3c3291490db58678522d356688 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:15.743196 12588 raft_consensus.cc:2932] T 70d4db3c3291490db58678522d356688 P 09ec2b380cbf4200af90b0eee891c010 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:15.745348 9367 catalog_manager.cc:5484] T 70d4db3c3291490db58678522d356688 P 8eb9945f989d4be6b51e1531a1365350 reported cstate change: config changed from index 8 to 9, VOTER 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68) evicted. New cstate: current_term: 1 leader_uuid: "1472c64688914149a318c7b8850a3df3" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:15.752667 9886 tablet_service.cc:1514] Processing DeleteTablet for tablet 70d4db3c3291490db58678522d356688 with delete_type TABLET_DATA_TOMBSTONED (TS 6cc5b9962ed24441bcba0c07b9f5438e not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:41940
I20240502 07:59:15.752991 16597 tablet_replica.cc:330] T 70d4db3c3291490db58678522d356688 P 6cc5b9962ed24441bcba0c07b9f5438e: stopping tablet replica
I20240502 07:59:15.753245 16597 raft_consensus.cc:2229] T 70d4db3c3291490db58678522d356688 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:15.753414 16597 raft_consensus.cc:2258] T 70d4db3c3291490db58678522d356688 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:15.753921 16597 ts_tablet_manager.cc:1875] T 70d4db3c3291490db58678522d356688 P 6cc5b9962ed24441bcba0c07b9f5438e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:15.755559 16597 ts_tablet_manager.cc:1888] T 70d4db3c3291490db58678522d356688 P 6cc5b9962ed24441bcba0c07b9f5438e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240502 07:59:15.755620 16597 log.cc:1196] T 70d4db3c3291490db58678522d356688 P 6cc5b9962ed24441bcba0c07b9f5438e: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal/wals/70d4db3c3291490db58678522d356688
I20240502 07:59:15.755987 9358 catalog_manager.cc:4830] TS 6cc5b9962ed24441bcba0c07b9f5438e (127.3.139.68:37379): tablet 70d4db3c3291490db58678522d356688 (table rebalance_test_table_9 [id=f0ea2346b9b9482e8343d199275d5819]) successfully deleted
I20240502 07:59:15.813141 11259 raft_consensus.cc:1203] T af07c527fff94115a743df1c73c3f4b5 P 51e82953ed2247fa9ff24638eab3508a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240502 07:59:15.930553 15531 raft_consensus.cc:1050] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a: attempting to promote NON_VOTER 09ec2b380cbf4200af90b0eee891c010 to VOTER
I20240502 07:59:15.930943 15531 consensus_queue.cc:237] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } }
I20240502 07:59:15.932299 12588 raft_consensus.cc:1261] T f7335ac0209c4e28b6751bb01d34d665 P 09ec2b380cbf4200af90b0eee891c010 [term 1 LEARNER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:15.932417 11259 raft_consensus.cc:1261] T f7335ac0209c4e28b6751bb01d34d665 P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:15.932439 9515 raft_consensus.cc:1261] T f7335ac0209c4e28b6751bb01d34d665 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:15.932845 16653 consensus_queue.cc:1035] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:15.933066 15531 consensus_queue.cc:1035] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:15.933321 16656 consensus_queue.cc:1035] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:15.935221 16653 raft_consensus.cc:2932] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:15.935722 9515 raft_consensus.cc:2932] T f7335ac0209c4e28b6751bb01d34d665 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:15.936156 12588 raft_consensus.cc:2932] T f7335ac0209c4e28b6751bb01d34d665 P 09ec2b380cbf4200af90b0eee891c010 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:15.937887 9369 catalog_manager.cc:5484] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a reported cstate change: config changed from index 7 to 8, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:15.937851 11260 raft_consensus.cc:2932] T f7335ac0209c4e28b6751bb01d34d665 P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:15.941210 9644 consensus_queue.cc:237] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } }
I20240502 07:59:15.942253 12589 raft_consensus.cc:1261] T f7335ac0209c4e28b6751bb01d34d665 P 09ec2b380cbf4200af90b0eee891c010 [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:15.942843 11260 raft_consensus.cc:1261] T f7335ac0209c4e28b6751bb01d34d665 P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:15.943351 16587 raft_consensus.cc:1050] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3: attempting to promote NON_VOTER 09ec2b380cbf4200af90b0eee891c010 to VOTER
I20240502 07:59:15.943308 16655 consensus_queue.cc:1035] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:15.943624 15531 consensus_queue.cc:1035] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:15.943746 16587 consensus_queue.cc:237] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } }
I20240502 07:59:15.944872 12589 raft_consensus.cc:1261] T 686b5cf6614d45c18cc776bd79567667 P 09ec2b380cbf4200af90b0eee891c010 [term 1 LEARNER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:15.944984 9515 raft_consensus.cc:1261] T 686b5cf6614d45c18cc776bd79567667 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:15.945387 11654 raft_consensus.cc:1261] T 686b5cf6614d45c18cc776bd79567667 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:15.945617 14872 consensus_queue.cc:1035] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:15.945952 14872 consensus_queue.cc:1035] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:15.945739 16656 raft_consensus.cc:2932] T f7335ac0209c4e28b6751bb01d34d665 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:15.946138 12588 raft_consensus.cc:2932] T f7335ac0209c4e28b6751bb01d34d665 P 09ec2b380cbf4200af90b0eee891c010 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:15.946406 16589 consensus_queue.cc:1035] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:15.948168 9358 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet f7335ac0209c4e28b6751bb01d34d665 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:15.949136 11260 raft_consensus.cc:2932] T f7335ac0209c4e28b6751bb01d34d665 P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:15.949234 16587 raft_consensus.cc:2932] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:15.949501 9367 catalog_manager.cc:5484] T f7335ac0209c4e28b6751bb01d34d665 P 09ec2b380cbf4200af90b0eee891c010 reported cstate change: config changed from index 8 to 9, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:15.950600 9515 raft_consensus.cc:2932] T 686b5cf6614d45c18cc776bd79567667 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:15.952483 11654 raft_consensus.cc:2932] T 686b5cf6614d45c18cc776bd79567667 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:15.952855 9369 catalog_manager.cc:5484] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 reported cstate change: config changed from index 7 to 8, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1472c64688914149a318c7b8850a3df3" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:15.953217 9493 tablet_service.cc:1514] Processing DeleteTablet for tablet f7335ac0209c4e28b6751bb01d34d665 with delete_type TABLET_DATA_TOMBSTONED (TS a201db721bda4dc3a4422dc8fa9bf72e not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:41730
I20240502 07:59:15.953228 12589 raft_consensus.cc:2932] T 686b5cf6614d45c18cc776bd79567667 P 09ec2b380cbf4200af90b0eee891c010 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:15.955230 16664 tablet_replica.cc:330] T f7335ac0209c4e28b6751bb01d34d665 P a201db721bda4dc3a4422dc8fa9bf72e: stopping tablet replica
I20240502 07:59:15.955402 16664 raft_consensus.cc:2229] T f7335ac0209c4e28b6751bb01d34d665 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:15.955663 16664 raft_consensus.cc:2258] T f7335ac0209c4e28b6751bb01d34d665 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:15.956586 9777 consensus_queue.cc:237] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } }
I20240502 07:59:15.957193 16664 ts_tablet_manager.cc:1875] T f7335ac0209c4e28b6751bb01d34d665 P a201db721bda4dc3a4422dc8fa9bf72e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:15.957739 12589 raft_consensus.cc:1261] T 686b5cf6614d45c18cc776bd79567667 P 09ec2b380cbf4200af90b0eee891c010 [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:15.957818 11654 raft_consensus.cc:1261] T 686b5cf6614d45c18cc776bd79567667 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Refusing update from remote peer 1472c64688914149a318c7b8850a3df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:15.958201 16589 consensus_queue.cc:1035] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:15.958397 16593 consensus_queue.cc:1035] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:15.959440 16664 ts_tablet_manager.cc:1888] T f7335ac0209c4e28b6751bb01d34d665 P a201db721bda4dc3a4422dc8fa9bf72e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240502 07:59:15.959568 16664 log.cc:1196] T f7335ac0209c4e28b6751bb01d34d665 P a201db721bda4dc3a4422dc8fa9bf72e: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal/wals/f7335ac0209c4e28b6751bb01d34d665
I20240502 07:59:15.959883 16593 raft_consensus.cc:2932] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:15.960435 9360 catalog_manager.cc:4830] TS a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551): tablet f7335ac0209c4e28b6751bb01d34d665 (table rebalance_test_table_7 [id=7100a319a6ec42499621cff56501ef8f]) successfully deleted
I20240502 07:59:15.960678 12589 raft_consensus.cc:2932] T 686b5cf6614d45c18cc776bd79567667 P 09ec2b380cbf4200af90b0eee891c010 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:15.960898 11654 raft_consensus.cc:2932] T 686b5cf6614d45c18cc776bd79567667 P a95f1d3af33f4c879fd290ca917d6329 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:15.962432 9360 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 686b5cf6614d45c18cc776bd79567667 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:15.962816 9369 catalog_manager.cc:5484] T 686b5cf6614d45c18cc776bd79567667 P 1472c64688914149a318c7b8850a3df3 reported cstate change: config changed from index 8 to 9, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New cstate: current_term: 1 leader_uuid: "1472c64688914149a318c7b8850a3df3" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a95f1d3af33f4c879fd290ca917d6329" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 36271 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:15.965592 9493 tablet_service.cc:1514] Processing DeleteTablet for tablet 686b5cf6614d45c18cc776bd79567667 with delete_type TABLET_DATA_TOMBSTONED (TS a201db721bda4dc3a4422dc8fa9bf72e not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:41730
I20240502 07:59:15.966214 16664 tablet_replica.cc:330] T 686b5cf6614d45c18cc776bd79567667 P a201db721bda4dc3a4422dc8fa9bf72e: stopping tablet replica
I20240502 07:59:15.966356 15356 raft_consensus.cc:1050] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e: attempting to promote NON_VOTER 09ec2b380cbf4200af90b0eee891c010 to VOTER
I20240502 07:59:15.966735 15356 consensus_queue.cc:237] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } }
I20240502 07:59:15.967988 16664 raft_consensus.cc:2229] T 686b5cf6614d45c18cc776bd79567667 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:15.968112 12589 raft_consensus.cc:1261] T eddeb6a23ec1418eb35b414f5d52e7b7 P 09ec2b380cbf4200af90b0eee891c010 [term 1 LEARNER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:15.968287 11391 raft_consensus.cc:1261] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:15.968288 9644 raft_consensus.cc:1261] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Refusing update from remote peer a201db721bda4dc3a4422dc8fa9bf72e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:15.968602 16666 consensus_queue.cc:1035] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:15.968781 16669 consensus_queue.cc:1035] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:15.968988 16664 raft_consensus.cc:2258] T 686b5cf6614d45c18cc776bd79567667 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:15.969094 16668 consensus_queue.cc:1035] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:15.970402 16664 ts_tablet_manager.cc:1875] T 686b5cf6614d45c18cc776bd79567667 P a201db721bda4dc3a4422dc8fa9bf72e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:15.970685 16666 raft_consensus.cc:2932] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:15.971086 12589 raft_consensus.cc:2932] T eddeb6a23ec1418eb35b414f5d52e7b7 P 09ec2b380cbf4200af90b0eee891c010 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:15.971762 9644 raft_consensus.cc:2932] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:15.972780 11391 raft_consensus.cc:2932] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:15.974503 16664 ts_tablet_manager.cc:1888] T 686b5cf6614d45c18cc776bd79567667 P a201db721bda4dc3a4422dc8fa9bf72e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240502 07:59:15.974593 16664 log.cc:1196] T 686b5cf6614d45c18cc776bd79567667 P a201db721bda4dc3a4422dc8fa9bf72e: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal/wals/686b5cf6614d45c18cc776bd79567667
I20240502 07:59:15.974555 9365 catalog_manager.cc:5484] T eddeb6a23ec1418eb35b414f5d52e7b7 P 09ec2b380cbf4200af90b0eee891c010 reported cstate change: config changed from index 7 to 8, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:15.975212 9360 catalog_manager.cc:4830] TS a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551): tablet 686b5cf6614d45c18cc776bd79567667 (table rebalance_test_table_0 [id=efdfc231272e43a5afefacf0ec75ec57]) successfully deleted
I20240502 07:59:16.102878 9515 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "eddeb6a23ec1418eb35b414f5d52e7b7"
dest_uuid: "a201db721bda4dc3a4422dc8fa9bf72e"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:43798
I20240502 07:59:16.103013 9515 raft_consensus.cc:595] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Received request to transfer leadership
I20240502 07:59:16.133425 9515 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "eddeb6a23ec1418eb35b414f5d52e7b7"
dest_uuid: "a201db721bda4dc3a4422dc8fa9bf72e"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:43824
I20240502 07:59:16.133560 9515 raft_consensus.cc:595] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Received request to transfer leadership
I20240502 07:59:16.217229 16723 raft_consensus.cc:1050] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e: attempting to promote NON_VOTER 51e82953ed2247fa9ff24638eab3508a to VOTER
I20240502 07:59:16.217736 16723 consensus_queue.cc:237] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } }
I20240502 07:59:16.219367 11260 raft_consensus.cc:1261] T af07c527fff94115a743df1c73c3f4b5 P 51e82953ed2247fa9ff24638eab3508a [term 1 LEARNER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:16.219421 11391 raft_consensus.cc:1261] T af07c527fff94115a743df1c73c3f4b5 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:16.219439 9777 raft_consensus.cc:1261] T af07c527fff94115a743df1c73c3f4b5 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:16.219815 16725 consensus_queue.cc:1035] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:16.220007 16723 consensus_queue.cc:1035] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:16.220189 16561 consensus_queue.cc:1035] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:16.222118 16725 raft_consensus.cc:2932] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:16.222924 9777 raft_consensus.cc:2932] T af07c527fff94115a743df1c73c3f4b5 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:16.224000 11391 raft_consensus.cc:2932] T af07c527fff94115a743df1c73c3f4b5 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:16.224822 11260 raft_consensus.cc:2932] T af07c527fff94115a743df1c73c3f4b5 P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:16.225646 9369 catalog_manager.cc:5484] T af07c527fff94115a743df1c73c3f4b5 P 1472c64688914149a318c7b8850a3df3 reported cstate change: config changed from index 7 to 8, 51e82953ed2247fa9ff24638eab3508a (127.3.139.69) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:16.228889 9908 consensus_queue.cc:237] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } }
I20240502 07:59:16.229852 11260 raft_consensus.cc:1261] T af07c527fff94115a743df1c73c3f4b5 P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:16.229861 11391 raft_consensus.cc:1261] T af07c527fff94115a743df1c73c3f4b5 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Refusing update from remote peer 6cc5b9962ed24441bcba0c07b9f5438e: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:16.230211 16561 consensus_queue.cc:1035] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:16.230394 16254 consensus_queue.cc:1035] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:16.231781 16723 raft_consensus.cc:2932] T af07c527fff94115a743df1c73c3f4b5 P 6cc5b9962ed24441bcba0c07b9f5438e [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:16.232228 11391 raft_consensus.cc:2932] T af07c527fff94115a743df1c73c3f4b5 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:16.233484 9358 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet af07c527fff94115a743df1c73c3f4b5 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:16.233517 11260 raft_consensus.cc:2932] T af07c527fff94115a743df1c73c3f4b5 P 51e82953ed2247fa9ff24638eab3508a [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:16.234265 9372 catalog_manager.cc:5484] T af07c527fff94115a743df1c73c3f4b5 P 8eb9945f989d4be6b51e1531a1365350 reported cstate change: config changed from index 8 to 9, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New cstate: current_term: 1 leader_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "6cc5b9962ed24441bcba0c07b9f5438e" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 37379 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "51e82953ed2247fa9ff24638eab3508a" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 45345 } attrs { promote: false } } }
I20240502 07:59:16.236815 9757 tablet_service.cc:1514] Processing DeleteTablet for tablet af07c527fff94115a743df1c73c3f4b5 with delete_type TABLET_DATA_TOMBSTONED (TS 1472c64688914149a318c7b8850a3df3 not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:57656
I20240502 07:59:16.237214 16730 tablet_replica.cc:330] T af07c527fff94115a743df1c73c3f4b5 P 1472c64688914149a318c7b8850a3df3: stopping tablet replica
I20240502 07:59:16.237357 16730 raft_consensus.cc:2229] T af07c527fff94115a743df1c73c3f4b5 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:16.237571 16730 raft_consensus.cc:2258] T af07c527fff94115a743df1c73c3f4b5 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:16.238236 16730 ts_tablet_manager.cc:1875] T af07c527fff94115a743df1c73c3f4b5 P 1472c64688914149a318c7b8850a3df3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:16.240047 16730 ts_tablet_manager.cc:1888] T af07c527fff94115a743df1c73c3f4b5 P 1472c64688914149a318c7b8850a3df3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240502 07:59:16.240113 16730 log.cc:1196] T af07c527fff94115a743df1c73c3f4b5 P 1472c64688914149a318c7b8850a3df3: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal/wals/af07c527fff94115a743df1c73c3f4b5
I20240502 07:59:16.240618 9360 catalog_manager.cc:4830] TS 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191): tablet af07c527fff94115a743df1c73c3f4b5 (table rebalance_test_table_6 [id=af6c4a01744d4fcfbdf53cfc8d19500b]) successfully deleted
I20240502 07:59:16.357223 9515 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "eddeb6a23ec1418eb35b414f5d52e7b7"
dest_uuid: "a201db721bda4dc3a4422dc8fa9bf72e"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:43840
I20240502 07:59:16.357353 9515 raft_consensus.cc:595] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Received request to transfer leadership
I20240502 07:59:16.381240 9515 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "eddeb6a23ec1418eb35b414f5d52e7b7"
dest_uuid: "a201db721bda4dc3a4422dc8fa9bf72e"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:43856
I20240502 07:59:16.381369 9515 raft_consensus.cc:595] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Received request to transfer leadership
I20240502 07:59:16.383666 16665 raft_consensus.cc:979] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e: : Instructing follower 09ec2b380cbf4200af90b0eee891c010 to start an election
I20240502 07:59:16.383752 16665 raft_consensus.cc:1067] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Signalling peer 09ec2b380cbf4200af90b0eee891c010 to start an election
I20240502 07:59:16.384189 12589 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "eddeb6a23ec1418eb35b414f5d52e7b7"
dest_uuid: "09ec2b380cbf4200af90b0eee891c010"
from {username='slave'} at 127.3.139.65:50785
I20240502 07:59:16.384343 12589 raft_consensus.cc:482] T eddeb6a23ec1418eb35b414f5d52e7b7 P 09ec2b380cbf4200af90b0eee891c010 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:16.384406 12589 raft_consensus.cc:3037] T eddeb6a23ec1418eb35b414f5d52e7b7 P 09ec2b380cbf4200af90b0eee891c010 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:16.385159 12589 raft_consensus.cc:504] T eddeb6a23ec1418eb35b414f5d52e7b7 P 09ec2b380cbf4200af90b0eee891c010 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } }
I20240502 07:59:16.386134 12589 leader_election.cc:288] T eddeb6a23ec1418eb35b414f5d52e7b7 P 09ec2b380cbf4200af90b0eee891c010 [CANDIDATE]: Term 2 election: Requested vote from peers 8337af1943564c0dad18b45a597b805a (127.3.139.66:46667), a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551), 8eb9945f989d4be6b51e1531a1365350 (127.3.139.70:45329)
I20240502 07:59:16.386503 9515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eddeb6a23ec1418eb35b414f5d52e7b7" candidate_uuid: "09ec2b380cbf4200af90b0eee891c010" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "a201db721bda4dc3a4422dc8fa9bf72e"
I20240502 07:59:16.386613 9515 raft_consensus.cc:3032] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:59:16.386659 9515 raft_consensus.cc:729] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a201db721bda4dc3a4422dc8fa9bf72e, State: Running, Role: LEADER
I20240502 07:59:16.386665 9644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eddeb6a23ec1418eb35b414f5d52e7b7" candidate_uuid: "09ec2b380cbf4200af90b0eee891c010" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "8337af1943564c0dad18b45a597b805a"
I20240502 07:59:16.386794 9644 raft_consensus.cc:3037] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8337af1943564c0dad18b45a597b805a [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:16.386785 9515 consensus_queue.cc:260] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } }
I20240502 07:59:16.386966 9515 raft_consensus.cc:3037] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:16.387583 9644 raft_consensus.cc:2454] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8337af1943564c0dad18b45a597b805a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09ec2b380cbf4200af90b0eee891c010 in term 2.
I20240502 07:59:16.387676 9515 raft_consensus.cc:2454] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09ec2b380cbf4200af90b0eee891c010 in term 2.
I20240502 07:59:16.387953 12523 leader_election.cc:302] T eddeb6a23ec1418eb35b414f5d52e7b7 P 09ec2b380cbf4200af90b0eee891c010 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 09ec2b380cbf4200af90b0eee891c010, 8337af1943564c0dad18b45a597b805a, a201db721bda4dc3a4422dc8fa9bf72e; no voters:
I20240502 07:59:16.388178 16414 raft_consensus.cc:2781] T eddeb6a23ec1418eb35b414f5d52e7b7 P 09ec2b380cbf4200af90b0eee891c010 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:16.388356 16414 raft_consensus.cc:686] T eddeb6a23ec1418eb35b414f5d52e7b7 P 09ec2b380cbf4200af90b0eee891c010 [term 2 LEADER]: Becoming Leader. State: Replica: 09ec2b380cbf4200af90b0eee891c010, State: Running, Role: LEADER
I20240502 07:59:16.388622 16414 consensus_queue.cc:237] T eddeb6a23ec1418eb35b414f5d52e7b7 P 09ec2b380cbf4200af90b0eee891c010 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } }
I20240502 07:59:16.390784 9367 catalog_manager.cc:5484] T eddeb6a23ec1418eb35b414f5d52e7b7 P 09ec2b380cbf4200af90b0eee891c010 reported cstate change: term changed from 1 to 2, leader changed from a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) to 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73). New cstate: current_term: 2 leader_uuid: "09ec2b380cbf4200af90b0eee891c010" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:16.391465 11391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eddeb6a23ec1418eb35b414f5d52e7b7" candidate_uuid: "09ec2b380cbf4200af90b0eee891c010" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "8eb9945f989d4be6b51e1531a1365350"
I20240502 07:59:16.391592 11391 raft_consensus.cc:3037] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8eb9945f989d4be6b51e1531a1365350 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:16.392436 11391 raft_consensus.cc:2454] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8eb9945f989d4be6b51e1531a1365350 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09ec2b380cbf4200af90b0eee891c010 in term 2.
I20240502 07:59:16.785077 11391 raft_consensus.cc:1261] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8eb9945f989d4be6b51e1531a1365350 [term 2 FOLLOWER]: Refusing update from remote peer 09ec2b380cbf4200af90b0eee891c010: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240502 07:59:16.785544 16414 consensus_queue.cc:1035] T eddeb6a23ec1418eb35b414f5d52e7b7 P 09ec2b380cbf4200af90b0eee891c010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:16.859692 9644 raft_consensus.cc:1261] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8337af1943564c0dad18b45a597b805a [term 2 FOLLOWER]: Refusing update from remote peer 09ec2b380cbf4200af90b0eee891c010: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240502 07:59:16.860116 16414 consensus_queue.cc:1035] T eddeb6a23ec1418eb35b414f5d52e7b7 P 09ec2b380cbf4200af90b0eee891c010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:16.862859 9515 raft_consensus.cc:1261] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [term 2 FOLLOWER]: Refusing update from remote peer 09ec2b380cbf4200af90b0eee891c010: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240502 07:59:16.863278 16768 consensus_queue.cc:1035] T eddeb6a23ec1418eb35b414f5d52e7b7 P 09ec2b380cbf4200af90b0eee891c010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a201db721bda4dc3a4422dc8fa9bf72e" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 39551 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:16.870666 12589 consensus_queue.cc:237] T eddeb6a23ec1418eb35b414f5d52e7b7 P 09ec2b380cbf4200af90b0eee891c010 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 8, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } }
I20240502 07:59:16.871693 11391 raft_consensus.cc:1261] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8eb9945f989d4be6b51e1531a1365350 [term 2 FOLLOWER]: Refusing update from remote peer 09ec2b380cbf4200af90b0eee891c010: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240502 07:59:16.871750 9644 raft_consensus.cc:1261] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8337af1943564c0dad18b45a597b805a [term 2 FOLLOWER]: Refusing update from remote peer 09ec2b380cbf4200af90b0eee891c010: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240502 07:59:16.872094 16791 consensus_queue.cc:1035] T eddeb6a23ec1418eb35b414f5d52e7b7 P 09ec2b380cbf4200af90b0eee891c010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:16.872243 16768 consensus_queue.cc:1035] T eddeb6a23ec1418eb35b414f5d52e7b7 P 09ec2b380cbf4200af90b0eee891c010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:16.873632 16414 raft_consensus.cc:2932] T eddeb6a23ec1418eb35b414f5d52e7b7 P 09ec2b380cbf4200af90b0eee891c010 [term 2 LEADER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:16.874122 11391 raft_consensus.cc:2932] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8eb9945f989d4be6b51e1531a1365350 [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:16.874138 9644 raft_consensus.cc:2932] T eddeb6a23ec1418eb35b414f5d52e7b7 P 8337af1943564c0dad18b45a597b805a [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:16.875161 9358 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet eddeb6a23ec1418eb35b414f5d52e7b7 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:16.876164 9367 catalog_manager.cc:5484] T eddeb6a23ec1418eb35b414f5d52e7b7 P 09ec2b380cbf4200af90b0eee891c010 reported cstate change: config changed from index 8 to 10, VOTER a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65) evicted. New cstate: current_term: 2 leader_uuid: "09ec2b380cbf4200af90b0eee891c010" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8eb9945f989d4be6b51e1531a1365350" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 45329 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:16.878388 9493 tablet_service.cc:1514] Processing DeleteTablet for tablet eddeb6a23ec1418eb35b414f5d52e7b7 with delete_type TABLET_DATA_TOMBSTONED (TS a201db721bda4dc3a4422dc8fa9bf72e not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:41730
I20240502 07:59:16.878691 16796 tablet_replica.cc:330] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e: stopping tablet replica
I20240502 07:59:16.878818 16796 raft_consensus.cc:2229] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:16.878923 16796 raft_consensus.cc:2258] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:16.879477 16796 ts_tablet_manager.cc:1875] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:16.881220 16796 ts_tablet_manager.cc:1888] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.9
I20240502 07:59:16.881291 16796 log.cc:1196] T eddeb6a23ec1418eb35b414f5d52e7b7 P a201db721bda4dc3a4422dc8fa9bf72e: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal/wals/eddeb6a23ec1418eb35b414f5d52e7b7
I20240502 07:59:16.881702 9360 catalog_manager.cc:4830] TS a201db721bda4dc3a4422dc8fa9bf72e (127.3.139.65:39551): tablet eddeb6a23ec1418eb35b414f5d52e7b7 (table rebalance_test_table_2 [id=1b8f60b832d543568624eb22602bb97c]) successfully deleted
I20240502 07:59:16.885991 16151 tablet_copy_client.cc:523] T ecd0995c31fe4e708712c62ca60511c9 P 09ec2b380cbf4200af90b0eee891c010: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:16.887112 16151 tablet_bootstrap.cc:492] T ecd0995c31fe4e708712c62ca60511c9 P 09ec2b380cbf4200af90b0eee891c010: Bootstrap starting.
I20240502 07:59:16.905042 16151 tablet_bootstrap.cc:492] T ecd0995c31fe4e708712c62ca60511c9 P 09ec2b380cbf4200af90b0eee891c010: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:16.905581 16151 tablet_bootstrap.cc:492] T ecd0995c31fe4e708712c62ca60511c9 P 09ec2b380cbf4200af90b0eee891c010: Bootstrap complete.
I20240502 07:59:16.905757 16151 ts_tablet_manager.cc:1367] T ecd0995c31fe4e708712c62ca60511c9 P 09ec2b380cbf4200af90b0eee891c010: Time spent bootstrapping tablet: real 0.019s user 0.019s sys 0.000s
I20240502 07:59:16.906159 16151 raft_consensus.cc:348] T ecd0995c31fe4e708712c62ca60511c9 P 09ec2b380cbf4200af90b0eee891c010 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } }
I20240502 07:59:16.906261 16151 raft_consensus.cc:729] T ecd0995c31fe4e708712c62ca60511c9 P 09ec2b380cbf4200af90b0eee891c010 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 09ec2b380cbf4200af90b0eee891c010, State: Initialized, Role: LEARNER
I20240502 07:59:16.906327 16151 consensus_queue.cc:260] T ecd0995c31fe4e708712c62ca60511c9 P 09ec2b380cbf4200af90b0eee891c010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: true } }
I20240502 07:59:16.906594 16151 ts_tablet_manager.cc:1398] T ecd0995c31fe4e708712c62ca60511c9 P 09ec2b380cbf4200af90b0eee891c010: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:16.907097 9666 tablet_copy_service.cc:342] P 8337af1943564c0dad18b45a597b805a: Request end of tablet copy session 09ec2b380cbf4200af90b0eee891c010-ecd0995c31fe4e708712c62ca60511c9 received from {username='slave'} at 127.3.139.73:58997
I20240502 07:59:16.907193 9666 tablet_copy_service.cc:433] P 8337af1943564c0dad18b45a597b805a: ending tablet copy session 09ec2b380cbf4200af90b0eee891c010-ecd0995c31fe4e708712c62ca60511c9 on tablet ecd0995c31fe4e708712c62ca60511c9 with peer 09ec2b380cbf4200af90b0eee891c010
I20240502 07:59:16.960109 12589 raft_consensus.cc:1203] T ecd0995c31fe4e708712c62ca60511c9 P 09ec2b380cbf4200af90b0eee891c010 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240502 07:59:17.579710 16824 raft_consensus.cc:1050] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a: attempting to promote NON_VOTER 09ec2b380cbf4200af90b0eee891c010 to VOTER
I20240502 07:59:17.580199 16824 consensus_queue.cc:237] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } }
I20240502 07:59:17.581449 12589 raft_consensus.cc:1261] T ecd0995c31fe4e708712c62ca60511c9 P 09ec2b380cbf4200af90b0eee891c010 [term 1 LEARNER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:17.581532 11520 raft_consensus.cc:1261] T ecd0995c31fe4e708712c62ca60511c9 P 98bef2e94ade4763a66c1ad68259744a [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:17.581538 9777 raft_consensus.cc:1261] T ecd0995c31fe4e708712c62ca60511c9 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:17.581857 16824 consensus_queue.cc:1035] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:17.582060 15531 consensus_queue.cc:1035] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:17.582253 16823 consensus_queue.cc:1035] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:17.584766 15531 raft_consensus.cc:2932] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:17.585152 9777 raft_consensus.cc:2932] T ecd0995c31fe4e708712c62ca60511c9 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:17.587697 9365 catalog_manager.cc:5484] T ecd0995c31fe4e708712c62ca60511c9 P 1472c64688914149a318c7b8850a3df3 reported cstate change: config changed from index 7 to 8, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:17.588574 12589 raft_consensus.cc:2932] T ecd0995c31fe4e708712c62ca60511c9 P 09ec2b380cbf4200af90b0eee891c010 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:17.592183 11520 raft_consensus.cc:2932] T ecd0995c31fe4e708712c62ca60511c9 P 98bef2e94ade4763a66c1ad68259744a [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 09ec2b380cbf4200af90b0eee891c010 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1472c64688914149a318c7b8850a3df3" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 43191 } attrs { replace: true } } peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:17.598935 9644 consensus_queue.cc:237] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } }
I20240502 07:59:17.600050 11520 raft_consensus.cc:1261] T ecd0995c31fe4e708712c62ca60511c9 P 98bef2e94ade4763a66c1ad68259744a [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:17.600051 12589 raft_consensus.cc:1261] T ecd0995c31fe4e708712c62ca60511c9 P 09ec2b380cbf4200af90b0eee891c010 [term 1 FOLLOWER]: Refusing update from remote peer 8337af1943564c0dad18b45a597b805a: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:17.600397 16826 consensus_queue.cc:1035] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:17.600579 15531 consensus_queue.cc:1035] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a [LEADER]: Connected to new peer: Peer: permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:17.602020 15531 raft_consensus.cc:2932] T ecd0995c31fe4e708712c62ca60511c9 P 8337af1943564c0dad18b45a597b805a [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:17.602774 11520 raft_consensus.cc:2932] T ecd0995c31fe4e708712c62ca60511c9 P 98bef2e94ade4763a66c1ad68259744a [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:17.605090 9358 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet ecd0995c31fe4e708712c62ca60511c9 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:17.605259 9367 catalog_manager.cc:5484] T ecd0995c31fe4e708712c62ca60511c9 P 98bef2e94ade4763a66c1ad68259744a reported cstate change: config changed from index 8 to 9, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New cstate: current_term: 1 leader_uuid: "8337af1943564c0dad18b45a597b805a" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:17.607940 9757 tablet_service.cc:1514] Processing DeleteTablet for tablet ecd0995c31fe4e708712c62ca60511c9 with delete_type TABLET_DATA_TOMBSTONED (TS 1472c64688914149a318c7b8850a3df3 not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:57656
I20240502 07:59:17.608269 16834 tablet_replica.cc:330] T ecd0995c31fe4e708712c62ca60511c9 P 1472c64688914149a318c7b8850a3df3: stopping tablet replica
I20240502 07:59:17.608407 16834 raft_consensus.cc:2229] T ecd0995c31fe4e708712c62ca60511c9 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:17.608630 16834 raft_consensus.cc:2258] T ecd0995c31fe4e708712c62ca60511c9 P 1472c64688914149a318c7b8850a3df3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:17.609251 16834 ts_tablet_manager.cc:1875] T ecd0995c31fe4e708712c62ca60511c9 P 1472c64688914149a318c7b8850a3df3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:17.611407 16834 ts_tablet_manager.cc:1888] T ecd0995c31fe4e708712c62ca60511c9 P 1472c64688914149a318c7b8850a3df3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240502 07:59:17.611482 16834 log.cc:1196] T ecd0995c31fe4e708712c62ca60511c9 P 1472c64688914149a318c7b8850a3df3: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal/wals/ecd0995c31fe4e708712c62ca60511c9
I20240502 07:59:17.611989 9360 catalog_manager.cc:4830] TS 1472c64688914149a318c7b8850a3df3 (127.3.139.67:43191): tablet ecd0995c31fe4e708712c62ca60511c9 (table rebalance_test_table_5 [id=51b1431ec18d4bb8b4d5ae1909ae1322]) successfully deleted
I20240502 07:59:17.612149 12589 raft_consensus.cc:2932] T ecd0995c31fe4e708712c62ca60511c9 P 09ec2b380cbf4200af90b0eee891c010 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 1472c64688914149a318c7b8850a3df3 (127.3.139.67) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8337af1943564c0dad18b45a597b805a" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 46667 } } peers { permanent_uuid: "98bef2e94ade4763a66c1ad68259744a" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 46577 } attrs { promote: false } } peers { permanent_uuid: "09ec2b380cbf4200af90b0eee891c010" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 35761 } attrs { promote: false } } }
I20240502 07:59:17.710206 12569 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:59:17.717692 9493 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:17.719425 11240 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:17.720536 9886 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:17.733225 11635 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:59:17.734704 9757 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:17.747526 11371 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:59:17.748800 9624 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:17.765086 11502 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:17.855917 9757 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:17.857589 9886 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:17.863340 11240 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:17.867326 12569 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:59:17.867911 11635 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:59:17.871341 9624 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:17.882012 9493 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:17.883175 11502 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:17.885340 11371 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:59:18.135304 12569 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:59:18.138010 9886 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:18.141196 11240 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:18.146030 9624 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:18.149238 11635 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:59:18.151897 11371 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:59:18.155419 9757 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:18.155786 9493 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:18.170794 11502 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:18.275578 12569 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:59:18.280731 9757 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:18.286944 9624 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:18.289402 11240 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:18.294016 11371 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:59:18.294140 9886 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:18.295683 11502 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:18.304276 9493 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:18.327466 11635 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:59:18.433808 12569 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:59:18.436726 9757 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:18.438943 11240 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:18.448366 9624 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:18.456274 11371 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:59:18.463357 11502 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:18.470527 9886 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:18.479195 9493 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:18.485389 11635 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:59:18.583369 9493 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:18.586611 12569 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:59:18.590845 11240 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:18.591883 11635 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:59:18.598052 9624 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:18.599027 9886 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:18.599522 11371 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:59:18.601135 9757 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:18.620859 11502 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
bcbc6b7ce1a14c38a0a71655f6b9fd9c | 127.3.139.126:46479 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.3.139.84:39259 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
09ec2b380cbf4200af90b0eee891c010 | 127.3.139.73:35761 | HEALTHY | <none> | 1 | 0
1472c64688914149a318c7b8850a3df3 | 127.3.139.67:43191 | HEALTHY | <none> | 5 | 0
51e82953ed2247fa9ff24638eab3508a | 127.3.139.69:45345 | HEALTHY | <none> | 0 | 0
6cc5b9962ed24441bcba0c07b9f5438e | 127.3.139.68:37379 | HEALTHY | <none> | 8 | 0
8337af1943564c0dad18b45a597b805a | 127.3.139.66:46667 | HEALTHY | <none> | 6 | 0
8eb9945f989d4be6b51e1531a1365350 | 127.3.139.70:45329 | HEALTHY | <none> | 2 | 0
98bef2e94ade4763a66c1ad68259744a | 127.3.139.71:46577 | HEALTHY | <none> | 0 | 0
a201db721bda4dc3a4422dc8fa9bf72e | 127.3.139.65:39551 | HEALTHY | <none> | 7 | 0
a95f1d3af33f4c879fd290ca917d6329 | 127.3.139.72:36271 | HEALTHY | <none> | 1 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 9
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
---------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_server_key_size | 768 | experimental | all 9 server(s) checked
local_ip_for_outbound_sockets | 127.3.139.65 | experimental | 127.3.139.65:39551
local_ip_for_outbound_sockets | 127.3.139.66 | experimental | 127.3.139.66:46667
local_ip_for_outbound_sockets | 127.3.139.67 | experimental | 127.3.139.67:43191
local_ip_for_outbound_sockets | 127.3.139.68 | experimental | 127.3.139.68:37379
local_ip_for_outbound_sockets | 127.3.139.69 | experimental | 127.3.139.69:45345
local_ip_for_outbound_sockets | 127.3.139.70 | experimental | 127.3.139.70:45329
local_ip_for_outbound_sockets | 127.3.139.71 | experimental | 127.3.139.71:46577
local_ip_for_outbound_sockets | 127.3.139.72 | experimental | 127.3.139.72:36271
local_ip_for_outbound_sockets | 127.3.139.73 | experimental | 127.3.139.73:35761
never_fsync | true | unsafe,advanced | all 9 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 9 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 9 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 9 server(s) checked
server_dump_info_format | pb | hidden | all 9 server(s) checked
server_dump_info_path | /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden | 127.3.139.65:39551
server_dump_info_path | /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.3.139.66:46667
server_dump_info_path | /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.3.139.67:43191
server_dump_info_path | /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.3.139.68:37379
server_dump_info_path | /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.3.139.69:45345
server_dump_info_path | /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.3.139.70:45329
server_dump_info_path | /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/data/info.pb | hidden | 127.3.139.71:46577
server_dump_info_path | /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/data/info.pb | hidden | 127.3.139.72:36271
server_dump_info_path | /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/data/info.pb | hidden | 127.3.139.73:35761
tablet_copy_download_file_inject_latency_ms | 1500 | hidden | all 9 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.3.139.84:39259 | all 9 server(s) checked
time_source | builtin | all 9 server(s) checked
Version Summary
Version | Servers
-----------------+--------------------------
1.18.0-SNAPSHOT | all 10 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
------------------------+----+---------+---------------+---------+------------+------------------+-------------
rebalance_test_table_0 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_1 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_2 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_3 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_4 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_5 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_6 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_7 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_8 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_9 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 10
First Quartile | 10
Median | 10
Third Quartile | 10
Maximum | 10
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 9
Tables | 10
Tablets | 30
Replicas | 90
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20240502 07:59:18.665827 3629 log_verifier.cc:126] Checking tablet 0e4efe560ef74763b80b70665c11d7f1
I20240502 07:59:18.690852 3629 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 0e4efe560ef74763b80b70665c11d7f1
I20240502 07:59:18.690917 3629 log_verifier.cc:126] Checking tablet 0fab37c27e184a2cacb08f05457fa6ae
I20240502 07:59:18.714418 3629 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 0fab37c27e184a2cacb08f05457fa6ae
I20240502 07:59:18.714476 3629 log_verifier.cc:126] Checking tablet 12e3bfe638d249c0934feda41b8a8876
I20240502 07:59:18.738288 3629 log_verifier.cc:177] Verified matching terms for 10 ops in tablet 12e3bfe638d249c0934feda41b8a8876
I20240502 07:59:18.738343 3629 log_verifier.cc:126] Checking tablet 140868fcb19a4e7c837ce0c2c1c7041e
I20240502 07:59:18.761469 3629 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 140868fcb19a4e7c837ce0c2c1c7041e
I20240502 07:59:18.761525 3629 log_verifier.cc:126] Checking tablet 16c2ef3c0f5c44b39327175f6519fb95
I20240502 07:59:18.785185 3629 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 16c2ef3c0f5c44b39327175f6519fb95
I20240502 07:59:18.785240 3629 log_verifier.cc:126] Checking tablet 265eca6fff644d83857ed48a04864a58
I20240502 07:59:18.808866 3629 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 265eca6fff644d83857ed48a04864a58
I20240502 07:59:18.808924 3629 log_verifier.cc:126] Checking tablet 3255d89d51a94426a78364230c9b249b
I20240502 07:59:18.832556 3629 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 3255d89d51a94426a78364230c9b249b
I20240502 07:59:18.832613 3629 log_verifier.cc:126] Checking tablet 364d227472894712a2ae15f4589a5a5b
I20240502 07:59:18.856712 3629 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 364d227472894712a2ae15f4589a5a5b
I20240502 07:59:18.856776 3629 log_verifier.cc:126] Checking tablet 427c037fa09c4b94b7bc6c71bdb61c2f
I20240502 07:59:18.880906 3629 log_verifier.cc:177] Verified matching terms for 7 ops in tablet 427c037fa09c4b94b7bc6c71bdb61c2f
I20240502 07:59:18.880970 3629 log_verifier.cc:126] Checking tablet 50f059668c6b4205a15d2c076e2789b0
I20240502 07:59:18.905074 3629 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 50f059668c6b4205a15d2c076e2789b0
I20240502 07:59:18.905140 3629 log_verifier.cc:126] Checking tablet 686b5cf6614d45c18cc776bd79567667
I20240502 07:59:18.929726 3629 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 686b5cf6614d45c18cc776bd79567667
I20240502 07:59:18.929793 3629 log_verifier.cc:126] Checking tablet 70d4db3c3291490db58678522d356688
I20240502 07:59:18.954958 3629 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 70d4db3c3291490db58678522d356688
I20240502 07:59:18.955026 3629 log_verifier.cc:126] Checking tablet 7da9d3b7fe8c40f6a68bb1b7a926d994
I20240502 07:59:18.979435 3629 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 7da9d3b7fe8c40f6a68bb1b7a926d994
I20240502 07:59:18.979496 3629 log_verifier.cc:126] Checking tablet 8ceb129e9cbe4c469e7dfba419719556
I20240502 07:59:19.003768 3629 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 8ceb129e9cbe4c469e7dfba419719556
I20240502 07:59:19.003826 3629 log_verifier.cc:126] Checking tablet 8cf1917de461431d98c5155db3eb2312
I20240502 07:59:19.027108 3629 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 8cf1917de461431d98c5155db3eb2312
I20240502 07:59:19.027164 3629 log_verifier.cc:126] Checking tablet 92f37161960e4ee894c75d9034434001
I20240502 07:59:19.051631 3629 log_verifier.cc:177] Verified matching terms for 10 ops in tablet 92f37161960e4ee894c75d9034434001
I20240502 07:59:19.051688 3629 log_verifier.cc:126] Checking tablet 96dab19eeaea4648913045482b3e5e9e
I20240502 07:59:19.076592 3629 log_verifier.cc:177] Verified matching terms for 7 ops in tablet 96dab19eeaea4648913045482b3e5e9e
I20240502 07:59:19.076656 3629 log_verifier.cc:126] Checking tablet a12a084dba23425996e9122c083c1444
I20240502 07:59:19.100528 3629 log_verifier.cc:177] Verified matching terms for 9 ops in tablet a12a084dba23425996e9122c083c1444
I20240502 07:59:19.100586 3629 log_verifier.cc:126] Checking tablet af07c527fff94115a743df1c73c3f4b5
I20240502 07:59:19.124193 3629 log_verifier.cc:177] Verified matching terms for 9 ops in tablet af07c527fff94115a743df1c73c3f4b5
I20240502 07:59:19.124248 3629 log_verifier.cc:126] Checking tablet bc896a65284c48cd864bb6b57dbe5ca0
I20240502 07:59:19.148042 3629 log_verifier.cc:177] Verified matching terms for 9 ops in tablet bc896a65284c48cd864bb6b57dbe5ca0
I20240502 07:59:19.148104 3629 log_verifier.cc:126] Checking tablet d37b8a7f016647a7a03baedc37ed5419
I20240502 07:59:19.172497 3629 log_verifier.cc:177] Verified matching terms for 9 ops in tablet d37b8a7f016647a7a03baedc37ed5419
I20240502 07:59:19.172564 3629 log_verifier.cc:126] Checking tablet d3db04c54a184eb8a1c23fec9e7145eb
I20240502 07:59:19.195971 3629 log_verifier.cc:177] Verified matching terms for 6 ops in tablet d3db04c54a184eb8a1c23fec9e7145eb
I20240502 07:59:19.196035 3629 log_verifier.cc:126] Checking tablet d562a250cee247a48e573b25ecf00429
I20240502 07:59:19.220491 3629 log_verifier.cc:177] Verified matching terms for 10 ops in tablet d562a250cee247a48e573b25ecf00429
I20240502 07:59:19.220551 3629 log_verifier.cc:126] Checking tablet e2ba8220f81a4d6cb3611ca53db91fe1
I20240502 07:59:19.243719 3629 log_verifier.cc:177] Verified matching terms for 6 ops in tablet e2ba8220f81a4d6cb3611ca53db91fe1
I20240502 07:59:19.243781 3629 log_verifier.cc:126] Checking tablet ecd0995c31fe4e708712c62ca60511c9
I20240502 07:59:19.268190 3629 log_verifier.cc:177] Verified matching terms for 9 ops in tablet ecd0995c31fe4e708712c62ca60511c9
I20240502 07:59:19.268249 3629 log_verifier.cc:126] Checking tablet eddeb6a23ec1418eb35b414f5d52e7b7
I20240502 07:59:19.292418 3629 log_verifier.cc:177] Verified matching terms for 10 ops in tablet eddeb6a23ec1418eb35b414f5d52e7b7
I20240502 07:59:19.292479 3629 log_verifier.cc:126] Checking tablet ee2e79e677f643409e3c8dd49b9517cf
I20240502 07:59:19.316959 3629 log_verifier.cc:177] Verified matching terms for 10 ops in tablet ee2e79e677f643409e3c8dd49b9517cf
I20240502 07:59:19.317026 3629 log_verifier.cc:126] Checking tablet f7335ac0209c4e28b6751bb01d34d665
I20240502 07:59:19.340883 3629 log_verifier.cc:177] Verified matching terms for 9 ops in tablet f7335ac0209c4e28b6751bb01d34d665
I20240502 07:59:19.340956 3629 log_verifier.cc:126] Checking tablet fcdf47a04bdd4b9fa600a0bba59a4542
I20240502 07:59:19.367624 3629 log_verifier.cc:177] Verified matching terms for 6 ops in tablet fcdf47a04bdd4b9fa600a0bba59a4542
I20240502 07:59:19.367681 3629 log_verifier.cc:126] Checking tablet fd251de859e049c38a632a17b2fbdeae
I20240502 07:59:19.391355 3629 log_verifier.cc:177] Verified matching terms for 6 ops in tablet fd251de859e049c38a632a17b2fbdeae
I20240502 07:59:19.391785 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 9433
W20240502 07:59:19.397836 11193 connection.cc:535] server connection from 127.3.139.65:36665 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20240502 07:59:19.398684 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 9564
I20240502 07:59:19.404903 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 9695
I20240502 07:59:19.410648 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 9826
I20240502 07:59:19.416633 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 11178
I20240502 07:59:19.421339 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 11309
I20240502 07:59:19.426541 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 11440
I20240502 07:59:19.431469 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 11571
I20240502 07:59:19.436764 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 12386
I20240502 07:59:19.441504 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 9342
2024-05-02T07:59:19Z chronyd exiting
[ OK ] TserverAddedDuringRebalancingTest.TserverStarts/1 (29355 ms)
[----------] 1 test from TserverAddedDuringRebalancingTest (29355 ms total)
[----------] 1 test from RebalancerAndSingleReplicaTablets
[ RUN ] RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove/5
I20240502 07:59:19.527455 3629 test_util.cc:268] Using random seed: -84381670
I20240502 07:59:19.528213 3629 ts_itest-base.cc:112] Starting cluster with:
I20240502 07:59:19.528251 3629 ts_itest-base.cc:113] --------------
I20240502 07:59:19.528261 3629 ts_itest-base.cc:114] 4 tablet servers
I20240502 07:59:19.528270 3629 ts_itest-base.cc:115] 1 replicas per TS
I20240502 07:59:19.528278 3629 ts_itest-base.cc:116] --------------
2024-05-02T07:59:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-02T07:59:19Z Disabled control of system clock
I20240502 07:59:19.536039 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.139.126:41843
--webserver_interface=127.3.139.126
--webserver_port=0
--builtin_ntp_servers=127.3.139.84:44509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.139.126:41843
--raft_prepare_replacement_before_eviction=true
--tserver_unresponsive_timeout_ms=3000
--enable_range_replica_placement=false with env {}
W20240502 07:59:19.634048 17110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:19.634227 17110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:59:19.634266 17110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:19.637143 17110 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:59:19.637197 17110 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240502 07:59:19.637233 17110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:59:19.637269 17110 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240502 07:59:19.637296 17110 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240502 07:59:19.640784 17110 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:44509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.3.139.126:41843
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.139.126:41843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.3.139.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.17110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:59:19.641479 17110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:59:19.642426 17110 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:59:19.648991 17118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:19.649415 17110 server_base.cc:1034] running on GCE node
W20240502 07:59:19.649732 17115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:19.650080 17116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:19.650513 17110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:59:19.651327 17110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:59:19.652487 17110 hybrid_clock.cc:648] HybridClock initialized: now 1714636759652473 us; error 28 us; skew 500 ppm
I20240502 07:59:19.654135 17110 webserver.cc:460] Webserver started at http://127.3.139.126:36519/ using document root <none> and password file <none>
I20240502 07:59:19.654614 17110 fs_manager.cc:359] Metadata directory not provided
I20240502 07:59:19.654670 17110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:59:19.654825 17110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:59:19.656314 17110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "0624d5d81dad41b68933fbeb4cf0e8dc"
format_stamp: "Formatted at 2024-05-02 07:59:19 on dist-test-slave-twcr"
I20240502 07:59:19.656741 17110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "0624d5d81dad41b68933fbeb4cf0e8dc"
format_stamp: "Formatted at 2024-05-02 07:59:19 on dist-test-slave-twcr"
I20240502 07:59:19.659662 17110 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 07:59:19.661569 17124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:19.662497 17110 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:19.662600 17110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal
uuid: "0624d5d81dad41b68933fbeb4cf0e8dc"
format_stamp: "Formatted at 2024-05-02 07:59:19 on dist-test-slave-twcr"
I20240502 07:59:19.662679 17110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:59:19.719280 17110 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:59:19.719873 17110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:59:19.720032 17110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:59:19.725908 17110 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.126:41843
I20240502 07:59:19.725934 17176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.126:41843 every 8 connection(s)
I20240502 07:59:19.727135 17110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240502 07:59:19.730171 17177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:19.730567 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 17110
I20240502 07:59:19.730686 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240502 07:59:19.736213 17177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0624d5d81dad41b68933fbeb4cf0e8dc: Bootstrap starting.
I20240502 07:59:19.738056 17177 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0624d5d81dad41b68933fbeb4cf0e8dc: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:19.738663 17177 log.cc:824] T 00000000000000000000000000000000 P 0624d5d81dad41b68933fbeb4cf0e8dc: Log is configured to *not* fsync() on all Append() calls
I20240502 07:59:19.740074 17177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0624d5d81dad41b68933fbeb4cf0e8dc: No bootstrap required, opened a new log
I20240502 07:59:19.742295 17177 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0624d5d81dad41b68933fbeb4cf0e8dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0624d5d81dad41b68933fbeb4cf0e8dc" member_type: VOTER last_known_addr { host: "127.3.139.126" port: 41843 } }
I20240502 07:59:19.742455 17177 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0624d5d81dad41b68933fbeb4cf0e8dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:19.742506 17177 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0624d5d81dad41b68933fbeb4cf0e8dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0624d5d81dad41b68933fbeb4cf0e8dc, State: Initialized, Role: FOLLOWER
I20240502 07:59:19.742914 17177 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0624d5d81dad41b68933fbeb4cf0e8dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0624d5d81dad41b68933fbeb4cf0e8dc" member_type: VOTER last_known_addr { host: "127.3.139.126" port: 41843 } }
I20240502 07:59:19.743083 17177 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0624d5d81dad41b68933fbeb4cf0e8dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 07:59:19.743147 17177 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0624d5d81dad41b68933fbeb4cf0e8dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 07:59:19.743229 17177 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0624d5d81dad41b68933fbeb4cf0e8dc [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:19.743814 17177 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0624d5d81dad41b68933fbeb4cf0e8dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0624d5d81dad41b68933fbeb4cf0e8dc" member_type: VOTER last_known_addr { host: "127.3.139.126" port: 41843 } }
I20240502 07:59:19.744083 17177 leader_election.cc:302] T 00000000000000000000000000000000 P 0624d5d81dad41b68933fbeb4cf0e8dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0624d5d81dad41b68933fbeb4cf0e8dc; no voters:
I20240502 07:59:19.744289 17177 leader_election.cc:288] T 00000000000000000000000000000000 P 0624d5d81dad41b68933fbeb4cf0e8dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 07:59:19.744621 17182 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0624d5d81dad41b68933fbeb4cf0e8dc [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:19.744796 17182 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0624d5d81dad41b68933fbeb4cf0e8dc [term 1 LEADER]: Becoming Leader. State: Replica: 0624d5d81dad41b68933fbeb4cf0e8dc, State: Running, Role: LEADER
I20240502 07:59:19.745043 17182 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0624d5d81dad41b68933fbeb4cf0e8dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0624d5d81dad41b68933fbeb4cf0e8dc" member_type: VOTER last_known_addr { host: "127.3.139.126" port: 41843 } }
I20240502 07:59:19.745204 17177 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0624d5d81dad41b68933fbeb4cf0e8dc [sys.catalog]: configured and running, proceeding with master startup.
I20240502 07:59:19.746505 17184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0624d5d81dad41b68933fbeb4cf0e8dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0624d5d81dad41b68933fbeb4cf0e8dc. Latest consensus state: current_term: 1 leader_uuid: "0624d5d81dad41b68933fbeb4cf0e8dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0624d5d81dad41b68933fbeb4cf0e8dc" member_type: VOTER last_known_addr { host: "127.3.139.126" port: 41843 } } }
I20240502 07:59:19.746502 17183 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0624d5d81dad41b68933fbeb4cf0e8dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0624d5d81dad41b68933fbeb4cf0e8dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0624d5d81dad41b68933fbeb4cf0e8dc" member_type: VOTER last_known_addr { host: "127.3.139.126" port: 41843 } } }
I20240502 07:59:19.746727 17183 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0624d5d81dad41b68933fbeb4cf0e8dc [sys.catalog]: This master's current role is: LEADER
I20240502 07:59:19.746727 17184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0624d5d81dad41b68933fbeb4cf0e8dc [sys.catalog]: This master's current role is: LEADER
W20240502 07:59:19.749414 17197 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0624d5d81dad41b68933fbeb4cf0e8dc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240502 07:59:19.749475 17197 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20240502 07:59:19.749667 17194 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240502 07:59:19.750317 17194 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240502 07:59:19.754274 17194 catalog_manager.cc:1347] Generated new cluster ID: c4fe91a25b68499492863826df935b71
I20240502 07:59:19.754333 17194 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240502 07:59:19.761035 17194 catalog_manager.cc:1370] Generated new certificate authority record
I20240502 07:59:19.762055 17194 catalog_manager.cc:1504] Loading token signing keys...
I20240502 07:59:19.774022 17194 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 0624d5d81dad41b68933fbeb4cf0e8dc: Generated new TSK 0
I20240502 07:59:19.774658 17194 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240502 07:59:19.782240 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.65:0
--local_ip_for_outbound_sockets=127.3.139.65
--webserver_interface=127.3.139.65
--webserver_port=0
--tserver_master_addrs=127.3.139.126:41843
--builtin_ntp_servers=127.3.139.84:44509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240502 07:59:19.880610 17201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:19.880810 17201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:59:19.880864 17201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:19.883773 17201 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:59:19.883832 17201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:59:19.883914 17201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.65
I20240502 07:59:19.887307 17201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:44509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.3.139.65
--webserver_port=0
--tserver_master_addrs=127.3.139.126:41843
--never_fsync=true
--heap_profile_path=/tmp/kudu.17201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.65
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:59:19.888063 17201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:59:19.889065 17201 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:59:19.897284 17206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:19.897648 17201 server_base.cc:1034] running on GCE node
W20240502 07:59:19.897938 17207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:19.898291 17209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:19.898660 17201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:59:19.899292 17201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:59:19.900452 17201 hybrid_clock.cc:648] HybridClock initialized: now 1714636759900428 us; error 37 us; skew 500 ppm
I20240502 07:59:19.902091 17201 webserver.cc:460] Webserver started at http://127.3.139.65:42155/ using document root <none> and password file <none>
I20240502 07:59:19.902578 17201 fs_manager.cc:359] Metadata directory not provided
I20240502 07:59:19.902630 17201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:59:19.902799 17201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:59:19.904263 17201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "428e758256a24cd78560bac584e16ab3"
format_stamp: "Formatted at 2024-05-02 07:59:19 on dist-test-slave-twcr"
I20240502 07:59:19.904697 17201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "428e758256a24cd78560bac584e16ab3"
format_stamp: "Formatted at 2024-05-02 07:59:19 on dist-test-slave-twcr"
I20240502 07:59:19.907560 17201 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 07:59:19.909444 17215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:19.910332 17201 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:19.910429 17201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "428e758256a24cd78560bac584e16ab3"
format_stamp: "Formatted at 2024-05-02 07:59:19 on dist-test-slave-twcr"
I20240502 07:59:19.910511 17201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:59:19.937023 17201 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:59:19.937506 17201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:59:19.937637 17201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:59:19.938076 17201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:59:19.938881 17201 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:59:19.938941 17201 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:19.939039 17201 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:59:19.939093 17201 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:19.947387 17201 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.65:32961
I20240502 07:59:19.947434 17328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.65:32961 every 8 connection(s)
I20240502 07:59:19.948175 17201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240502 07:59:19.955581 17329 heartbeater.cc:347] Connected to a master server at 127.3.139.126:41843
I20240502 07:59:19.955757 17329 heartbeater.cc:464] Registering TS with master...
I20240502 07:59:19.956197 17329 heartbeater.cc:511] Master 127.3.139.126:41843 requested a full tablet report, sending...
I20240502 07:59:19.957172 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 17201
I20240502 07:59:19.957319 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240502 07:59:19.957553 17141 ts_manager.cc:194] Registered new tserver with Master: 428e758256a24cd78560bac584e16ab3 (127.3.139.65:32961)
I20240502 07:59:19.958760 17141 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.65:51001
I20240502 07:59:19.959491 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.66:0
--local_ip_for_outbound_sockets=127.3.139.66
--webserver_interface=127.3.139.66
--webserver_port=0
--tserver_master_addrs=127.3.139.126:41843
--builtin_ntp_servers=127.3.139.84:44509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240502 07:59:20.057009 17332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:20.057171 17332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:59:20.057210 17332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:20.060338 17332 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:59:20.060398 17332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:59:20.060492 17332 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.66
I20240502 07:59:20.063984 17332 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:44509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.3.139.66
--webserver_port=0
--tserver_master_addrs=127.3.139.126:41843
--never_fsync=true
--heap_profile_path=/tmp/kudu.17332
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.66
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:59:20.064714 17332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:59:20.065681 17332 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:59:20.073176 17340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:20.073536 17337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:20.073933 17338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:20.074335 17332 server_base.cc:1034] running on GCE node
I20240502 07:59:20.074633 17332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:59:20.075134 17332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:59:20.076269 17332 hybrid_clock.cc:648] HybridClock initialized: now 1714636760076258 us; error 24 us; skew 500 ppm
I20240502 07:59:20.077841 17332 webserver.cc:460] Webserver started at http://127.3.139.66:42947/ using document root <none> and password file <none>
I20240502 07:59:20.078322 17332 fs_manager.cc:359] Metadata directory not provided
I20240502 07:59:20.078372 17332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:59:20.078529 17332 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:59:20.080061 17332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "7fa0cca78aa4498e9ba57b22880c9573"
format_stamp: "Formatted at 2024-05-02 07:59:20 on dist-test-slave-twcr"
I20240502 07:59:20.080475 17332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "7fa0cca78aa4498e9ba57b22880c9573"
format_stamp: "Formatted at 2024-05-02 07:59:20 on dist-test-slave-twcr"
I20240502 07:59:20.083391 17332 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 07:59:20.085253 17346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:20.086118 17332 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:20.086210 17332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "7fa0cca78aa4498e9ba57b22880c9573"
format_stamp: "Formatted at 2024-05-02 07:59:20 on dist-test-slave-twcr"
I20240502 07:59:20.086282 17332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:59:20.106949 17332 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:59:20.107398 17332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:59:20.107525 17332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:59:20.107929 17332 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:59:20.108649 17332 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:59:20.108700 17332 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:20.108744 17332 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:59:20.108770 17332 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:20.116501 17332 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.66:45373
I20240502 07:59:20.116544 17459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.66:45373 every 8 connection(s)
I20240502 07:59:20.117261 17332 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240502 07:59:20.124019 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 17332
I20240502 07:59:20.124132 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240502 07:59:20.124431 17460 heartbeater.cc:347] Connected to a master server at 127.3.139.126:41843
I20240502 07:59:20.124620 17460 heartbeater.cc:464] Registering TS with master...
I20240502 07:59:20.125105 17460 heartbeater.cc:511] Master 127.3.139.126:41843 requested a full tablet report, sending...
I20240502 07:59:20.125952 17141 ts_manager.cc:194] Registered new tserver with Master: 7fa0cca78aa4498e9ba57b22880c9573 (127.3.139.66:45373)
I20240502 07:59:20.125982 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.67:0
--local_ip_for_outbound_sockets=127.3.139.67
--webserver_interface=127.3.139.67
--webserver_port=0
--tserver_master_addrs=127.3.139.126:41843
--builtin_ntp_servers=127.3.139.84:44509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 07:59:20.126467 17141 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.66:46321
W20240502 07:59:20.224489 17463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:20.224658 17463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:59:20.224699 17463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:20.227535 17463 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:59:20.227584 17463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:59:20.227645 17463 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.67
I20240502 07:59:20.231014 17463 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:44509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.3.139.67
--webserver_port=0
--tserver_master_addrs=127.3.139.126:41843
--never_fsync=true
--heap_profile_path=/tmp/kudu.17463
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.67
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:59:20.231698 17463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:59:20.232617 17463 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:59:20.239780 17469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:20.240123 17471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:20.240515 17468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:20.240885 17463 server_base.cc:1034] running on GCE node
I20240502 07:59:20.241219 17463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:59:20.241683 17463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:59:20.242820 17463 hybrid_clock.cc:648] HybridClock initialized: now 1714636760242808 us; error 31 us; skew 500 ppm
I20240502 07:59:20.244423 17463 webserver.cc:460] Webserver started at http://127.3.139.67:41581/ using document root <none> and password file <none>
I20240502 07:59:20.244946 17463 fs_manager.cc:359] Metadata directory not provided
I20240502 07:59:20.245002 17463 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:59:20.245153 17463 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:59:20.246659 17463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "b9991ebec9ab49ee88c8dd2cacfd6c56"
format_stamp: "Formatted at 2024-05-02 07:59:20 on dist-test-slave-twcr"
I20240502 07:59:20.247119 17463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "b9991ebec9ab49ee88c8dd2cacfd6c56"
format_stamp: "Formatted at 2024-05-02 07:59:20 on dist-test-slave-twcr"
I20240502 07:59:20.249917 17463 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:20.251863 17477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:20.252709 17463 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:20.252813 17463 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "b9991ebec9ab49ee88c8dd2cacfd6c56"
format_stamp: "Formatted at 2024-05-02 07:59:20 on dist-test-slave-twcr"
I20240502 07:59:20.252884 17463 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:59:20.262598 17463 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:59:20.263044 17463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:59:20.263195 17463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:59:20.263583 17463 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:59:20.264307 17463 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:59:20.264350 17463 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:20.264384 17463 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:59:20.264403 17463 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:20.272291 17463 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.67:36601
I20240502 07:59:20.272318 17590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.67:36601 every 8 connection(s)
I20240502 07:59:20.273042 17463 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240502 07:59:20.280325 17591 heartbeater.cc:347] Connected to a master server at 127.3.139.126:41843
I20240502 07:59:20.280491 17591 heartbeater.cc:464] Registering TS with master...
I20240502 07:59:20.280902 17591 heartbeater.cc:511] Master 127.3.139.126:41843 requested a full tablet report, sending...
I20240502 07:59:20.281157 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 17463
I20240502 07:59:20.281252 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240502 07:59:20.281893 17141 ts_manager.cc:194] Registered new tserver with Master: b9991ebec9ab49ee88c8dd2cacfd6c56 (127.3.139.67:36601)
I20240502 07:59:20.282392 17141 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.67:33903
I20240502 07:59:20.283138 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.68:0
--local_ip_for_outbound_sockets=127.3.139.68
--webserver_interface=127.3.139.68
--webserver_port=0
--tserver_master_addrs=127.3.139.126:41843
--builtin_ntp_servers=127.3.139.84:44509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240502 07:59:20.379384 17594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:20.379567 17594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:59:20.379608 17594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:20.382520 17594 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:59:20.382575 17594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:59:20.382650 17594 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.68
I20240502 07:59:20.386023 17594 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:44509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.68:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.3.139.68
--webserver_port=0
--tserver_master_addrs=127.3.139.126:41843
--never_fsync=true
--heap_profile_path=/tmp/kudu.17594
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.68
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:59:20.386744 17594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:59:20.387744 17594 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:59:20.395345 17602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:20.395659 17599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:20.396010 17600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:20.396387 17594 server_base.cc:1034] running on GCE node
I20240502 07:59:20.396696 17594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:59:20.397207 17594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:59:20.398352 17594 hybrid_clock.cc:648] HybridClock initialized: now 1714636760398338 us; error 24 us; skew 500 ppm
I20240502 07:59:20.400053 17594 webserver.cc:460] Webserver started at http://127.3.139.68:39493/ using document root <none> and password file <none>
I20240502 07:59:20.400641 17594 fs_manager.cc:359] Metadata directory not provided
I20240502 07:59:20.400702 17594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:59:20.400861 17594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:59:20.402343 17594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "db1de15a6fb94d6f83895fc5f1b6c809"
format_stamp: "Formatted at 2024-05-02 07:59:20 on dist-test-slave-twcr"
I20240502 07:59:20.402783 17594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "db1de15a6fb94d6f83895fc5f1b6c809"
format_stamp: "Formatted at 2024-05-02 07:59:20 on dist-test-slave-twcr"
I20240502 07:59:20.405879 17594 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 07:59:20.407928 17608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:20.408830 17594 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:20.408923 17594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "db1de15a6fb94d6f83895fc5f1b6c809"
format_stamp: "Formatted at 2024-05-02 07:59:20 on dist-test-slave-twcr"
I20240502 07:59:20.409003 17594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:59:20.428881 17594 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:59:20.429328 17594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:59:20.429461 17594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:59:20.429889 17594 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:59:20.430639 17594 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:59:20.430681 17594 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:20.430727 17594 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:59:20.430759 17594 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:20.438167 17594 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.68:33975
I20240502 07:59:20.438187 17721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.68:33975 every 8 connection(s)
I20240502 07:59:20.438930 17594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240502 07:59:20.446528 17722 heartbeater.cc:347] Connected to a master server at 127.3.139.126:41843
I20240502 07:59:20.446695 17722 heartbeater.cc:464] Registering TS with master...
I20240502 07:59:20.447207 17722 heartbeater.cc:511] Master 127.3.139.126:41843 requested a full tablet report, sending...
I20240502 07:59:20.447587 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 17594
I20240502 07:59:20.447703 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240502 07:59:20.448091 17141 ts_manager.cc:194] Registered new tserver with Master: db1de15a6fb94d6f83895fc5f1b6c809 (127.3.139.68:33975)
I20240502 07:59:20.448689 17141 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.68:54873
I20240502 07:59:20.449839 3629 external_mini_cluster.cc:930] 4 TS(s) registered with all masters
I20240502 07:59:20.458173 17141 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48990:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20240502 07:59:20.471877 17263 tablet_service.cc:1467] Processing CreateTablet for tablet 57b9f7ff4cd247c9a6d81e3028a1810f (DEFAULT_TABLE table=TestTable [id=6f1c0e11315a47079db380207b5f0e38]), partition=RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:20.472648 17263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57b9f7ff4cd247c9a6d81e3028a1810f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:20.477152 17735 tablet_bootstrap.cc:492] T 57b9f7ff4cd247c9a6d81e3028a1810f P 428e758256a24cd78560bac584e16ab3: Bootstrap starting.
I20240502 07:59:20.479025 17735 tablet_bootstrap.cc:654] T 57b9f7ff4cd247c9a6d81e3028a1810f P 428e758256a24cd78560bac584e16ab3: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:20.479672 17735 log.cc:824] T 57b9f7ff4cd247c9a6d81e3028a1810f P 428e758256a24cd78560bac584e16ab3: Log is configured to *not* fsync() on all Append() calls
I20240502 07:59:20.481040 17735 tablet_bootstrap.cc:492] T 57b9f7ff4cd247c9a6d81e3028a1810f P 428e758256a24cd78560bac584e16ab3: No bootstrap required, opened a new log
I20240502 07:59:20.481204 17735 ts_tablet_manager.cc:1367] T 57b9f7ff4cd247c9a6d81e3028a1810f P 428e758256a24cd78560bac584e16ab3: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240502 07:59:20.484078 17735 raft_consensus.cc:348] T 57b9f7ff4cd247c9a6d81e3028a1810f P 428e758256a24cd78560bac584e16ab3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "428e758256a24cd78560bac584e16ab3" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 32961 } }
I20240502 07:59:20.484233 17735 raft_consensus.cc:374] T 57b9f7ff4cd247c9a6d81e3028a1810f P 428e758256a24cd78560bac584e16ab3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:20.484279 17735 raft_consensus.cc:729] T 57b9f7ff4cd247c9a6d81e3028a1810f P 428e758256a24cd78560bac584e16ab3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 428e758256a24cd78560bac584e16ab3, State: Initialized, Role: FOLLOWER
I20240502 07:59:20.484658 17735 consensus_queue.cc:260] T 57b9f7ff4cd247c9a6d81e3028a1810f P 428e758256a24cd78560bac584e16ab3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "428e758256a24cd78560bac584e16ab3" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 32961 } }
I20240502 07:59:20.484781 17735 raft_consensus.cc:388] T 57b9f7ff4cd247c9a6d81e3028a1810f P 428e758256a24cd78560bac584e16ab3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 07:59:20.484840 17735 raft_consensus.cc:482] T 57b9f7ff4cd247c9a6d81e3028a1810f P 428e758256a24cd78560bac584e16ab3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 07:59:20.484939 17735 raft_consensus.cc:3037] T 57b9f7ff4cd247c9a6d81e3028a1810f P 428e758256a24cd78560bac584e16ab3 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:20.485638 17735 raft_consensus.cc:504] T 57b9f7ff4cd247c9a6d81e3028a1810f P 428e758256a24cd78560bac584e16ab3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "428e758256a24cd78560bac584e16ab3" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 32961 } }
I20240502 07:59:20.485934 17735 leader_election.cc:302] T 57b9f7ff4cd247c9a6d81e3028a1810f P 428e758256a24cd78560bac584e16ab3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 428e758256a24cd78560bac584e16ab3; no voters:
I20240502 07:59:20.486189 17735 leader_election.cc:288] T 57b9f7ff4cd247c9a6d81e3028a1810f P 428e758256a24cd78560bac584e16ab3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 07:59:20.486263 17737 raft_consensus.cc:2781] T 57b9f7ff4cd247c9a6d81e3028a1810f P 428e758256a24cd78560bac584e16ab3 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:20.486459 17737 raft_consensus.cc:686] T 57b9f7ff4cd247c9a6d81e3028a1810f P 428e758256a24cd78560bac584e16ab3 [term 1 LEADER]: Becoming Leader. State: Replica: 428e758256a24cd78560bac584e16ab3, State: Running, Role: LEADER
I20240502 07:59:20.486730 17737 consensus_queue.cc:237] T 57b9f7ff4cd247c9a6d81e3028a1810f P 428e758256a24cd78560bac584e16ab3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "428e758256a24cd78560bac584e16ab3" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 32961 } }
I20240502 07:59:20.486791 17329 heartbeater.cc:503] Master 127.3.139.126:41843 was elected leader, sending a full tablet report...
I20240502 07:59:20.486965 17735 ts_tablet_manager.cc:1398] T 57b9f7ff4cd247c9a6d81e3028a1810f P 428e758256a24cd78560bac584e16ab3: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20240502 07:59:20.489756 17141 catalog_manager.cc:5484] T 57b9f7ff4cd247c9a6d81e3028a1810f P 428e758256a24cd78560bac584e16ab3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 428e758256a24cd78560bac584e16ab3 (127.3.139.65). New cstate: current_term: 1 leader_uuid: "428e758256a24cd78560bac584e16ab3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "428e758256a24cd78560bac584e16ab3" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 32961 } health_report { overall_health: HEALTHY } } }
I20240502 07:59:20.498191 3629 external_mini_cluster.cc:930] 4 TS(s) registered with all masters
I20240502 07:59:20.499377 3629 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 428e758256a24cd78560bac584e16ab3 to finish bootstrapping
I20240502 07:59:20.504231 3629 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 7fa0cca78aa4498e9ba57b22880c9573 to finish bootstrapping
I20240502 07:59:20.509459 3629 ts_itest-base.cc:233] Waiting for 0 tablets on tserver b9991ebec9ab49ee88c8dd2cacfd6c56 to finish bootstrapping
I20240502 07:59:20.514536 3629 ts_itest-base.cc:233] Waiting for 0 tablets on tserver db1de15a6fb94d6f83895fc5f1b6c809 to finish bootstrapping
I20240502 07:59:20.519125 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 17463
I20240502 07:59:20.523739 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 17594
I20240502 07:59:21.128199 17460 heartbeater.cc:503] Master 127.3.139.126:41843 was elected leader, sending a full tablet report...
I20240502 07:59:24.279321 17141 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48990:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:59:24.285830 17263 tablet_service.cc:1467] Processing CreateTablet for tablet bff03e3cb849492f85ee3339da4a5dfa (DEFAULT_TABLE table=rebalance_test_table_0 [id=1a89ec9b6fe14c239932ac7dc8975687]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:24.286139 17263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bff03e3cb849492f85ee3339da4a5dfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:24.288302 17752 tablet_bootstrap.cc:492] T bff03e3cb849492f85ee3339da4a5dfa P 428e758256a24cd78560bac584e16ab3: Bootstrap starting.
I20240502 07:59:24.289196 17752 tablet_bootstrap.cc:654] T bff03e3cb849492f85ee3339da4a5dfa P 428e758256a24cd78560bac584e16ab3: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:24.290232 17752 tablet_bootstrap.cc:492] T bff03e3cb849492f85ee3339da4a5dfa P 428e758256a24cd78560bac584e16ab3: No bootstrap required, opened a new log
I20240502 07:59:24.290565 17752 ts_tablet_manager.cc:1367] T bff03e3cb849492f85ee3339da4a5dfa P 428e758256a24cd78560bac584e16ab3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:24.290684 17394 tablet_service.cc:1467] Processing CreateTablet for tablet 276002f558e54354897464ea6e718c20 (DEFAULT_TABLE table=rebalance_test_table_0 [id=1a89ec9b6fe14c239932ac7dc8975687]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:24.290692 17393 tablet_service.cc:1467] Processing CreateTablet for tablet 385590337e9d443a825b8a78922cda2a (DEFAULT_TABLE table=rebalance_test_table_0 [id=1a89ec9b6fe14c239932ac7dc8975687]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:24.291159 17752 raft_consensus.cc:348] T bff03e3cb849492f85ee3339da4a5dfa P 428e758256a24cd78560bac584e16ab3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "428e758256a24cd78560bac584e16ab3" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 32961 } }
I20240502 07:59:24.291254 17752 raft_consensus.cc:374] T bff03e3cb849492f85ee3339da4a5dfa P 428e758256a24cd78560bac584e16ab3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:24.291278 17752 raft_consensus.cc:729] T bff03e3cb849492f85ee3339da4a5dfa P 428e758256a24cd78560bac584e16ab3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 428e758256a24cd78560bac584e16ab3, State: Initialized, Role: FOLLOWER
I20240502 07:59:24.291400 17752 consensus_queue.cc:260] T bff03e3cb849492f85ee3339da4a5dfa P 428e758256a24cd78560bac584e16ab3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "428e758256a24cd78560bac584e16ab3" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 32961 } }
I20240502 07:59:24.291481 17752 raft_consensus.cc:388] T bff03e3cb849492f85ee3339da4a5dfa P 428e758256a24cd78560bac584e16ab3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 07:59:24.291512 17752 raft_consensus.cc:482] T bff03e3cb849492f85ee3339da4a5dfa P 428e758256a24cd78560bac584e16ab3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 07:59:24.291549 17752 raft_consensus.cc:3037] T bff03e3cb849492f85ee3339da4a5dfa P 428e758256a24cd78560bac584e16ab3 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:24.291566 17394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 276002f558e54354897464ea6e718c20. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:24.291831 17393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 385590337e9d443a825b8a78922cda2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:24.292404 17752 raft_consensus.cc:504] T bff03e3cb849492f85ee3339da4a5dfa P 428e758256a24cd78560bac584e16ab3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "428e758256a24cd78560bac584e16ab3" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 32961 } }
I20240502 07:59:24.292528 17752 leader_election.cc:302] T bff03e3cb849492f85ee3339da4a5dfa P 428e758256a24cd78560bac584e16ab3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 428e758256a24cd78560bac584e16ab3; no voters:
I20240502 07:59:24.292672 17752 leader_election.cc:288] T bff03e3cb849492f85ee3339da4a5dfa P 428e758256a24cd78560bac584e16ab3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 07:59:24.292748 17754 raft_consensus.cc:2781] T bff03e3cb849492f85ee3339da4a5dfa P 428e758256a24cd78560bac584e16ab3 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:24.292905 17754 raft_consensus.cc:686] T bff03e3cb849492f85ee3339da4a5dfa P 428e758256a24cd78560bac584e16ab3 [term 1 LEADER]: Becoming Leader. State: Replica: 428e758256a24cd78560bac584e16ab3, State: Running, Role: LEADER
I20240502 07:59:24.292932 17752 ts_tablet_manager.cc:1398] T bff03e3cb849492f85ee3339da4a5dfa P 428e758256a24cd78560bac584e16ab3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:24.293022 17754 consensus_queue.cc:237] T bff03e3cb849492f85ee3339da4a5dfa P 428e758256a24cd78560bac584e16ab3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "428e758256a24cd78560bac584e16ab3" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 32961 } }
I20240502 07:59:24.294224 17141 catalog_manager.cc:5484] T bff03e3cb849492f85ee3339da4a5dfa P 428e758256a24cd78560bac584e16ab3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 428e758256a24cd78560bac584e16ab3 (127.3.139.65). New cstate: current_term: 1 leader_uuid: "428e758256a24cd78560bac584e16ab3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "428e758256a24cd78560bac584e16ab3" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 32961 } health_report { overall_health: HEALTHY } } }
I20240502 07:59:24.297030 17759 tablet_bootstrap.cc:492] T 385590337e9d443a825b8a78922cda2a P 7fa0cca78aa4498e9ba57b22880c9573: Bootstrap starting.
I20240502 07:59:24.298725 17759 tablet_bootstrap.cc:654] T 385590337e9d443a825b8a78922cda2a P 7fa0cca78aa4498e9ba57b22880c9573: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:24.299348 17759 log.cc:824] T 385590337e9d443a825b8a78922cda2a P 7fa0cca78aa4498e9ba57b22880c9573: Log is configured to *not* fsync() on all Append() calls
I20240502 07:59:24.300634 17759 tablet_bootstrap.cc:492] T 385590337e9d443a825b8a78922cda2a P 7fa0cca78aa4498e9ba57b22880c9573: No bootstrap required, opened a new log
I20240502 07:59:24.300822 17759 ts_tablet_manager.cc:1367] T 385590337e9d443a825b8a78922cda2a P 7fa0cca78aa4498e9ba57b22880c9573: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20240502 07:59:24.303465 17759 raft_consensus.cc:348] T 385590337e9d443a825b8a78922cda2a P 7fa0cca78aa4498e9ba57b22880c9573 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fa0cca78aa4498e9ba57b22880c9573" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45373 } }
I20240502 07:59:24.303609 17759 raft_consensus.cc:374] T 385590337e9d443a825b8a78922cda2a P 7fa0cca78aa4498e9ba57b22880c9573 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:24.303654 17759 raft_consensus.cc:729] T 385590337e9d443a825b8a78922cda2a P 7fa0cca78aa4498e9ba57b22880c9573 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fa0cca78aa4498e9ba57b22880c9573, State: Initialized, Role: FOLLOWER
I20240502 07:59:24.303993 17759 consensus_queue.cc:260] T 385590337e9d443a825b8a78922cda2a P 7fa0cca78aa4498e9ba57b22880c9573 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fa0cca78aa4498e9ba57b22880c9573" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45373 } }
I20240502 07:59:24.304095 17759 raft_consensus.cc:388] T 385590337e9d443a825b8a78922cda2a P 7fa0cca78aa4498e9ba57b22880c9573 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 07:59:24.304149 17759 raft_consensus.cc:482] T 385590337e9d443a825b8a78922cda2a P 7fa0cca78aa4498e9ba57b22880c9573 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 07:59:24.304244 17759 raft_consensus.cc:3037] T 385590337e9d443a825b8a78922cda2a P 7fa0cca78aa4498e9ba57b22880c9573 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:24.304862 17759 raft_consensus.cc:504] T 385590337e9d443a825b8a78922cda2a P 7fa0cca78aa4498e9ba57b22880c9573 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fa0cca78aa4498e9ba57b22880c9573" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45373 } }
I20240502 07:59:24.305130 17759 leader_election.cc:302] T 385590337e9d443a825b8a78922cda2a P 7fa0cca78aa4498e9ba57b22880c9573 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7fa0cca78aa4498e9ba57b22880c9573; no voters:
I20240502 07:59:24.305359 17759 leader_election.cc:288] T 385590337e9d443a825b8a78922cda2a P 7fa0cca78aa4498e9ba57b22880c9573 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 07:59:24.305430 17761 raft_consensus.cc:2781] T 385590337e9d443a825b8a78922cda2a P 7fa0cca78aa4498e9ba57b22880c9573 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:24.305869 17761 raft_consensus.cc:686] T 385590337e9d443a825b8a78922cda2a P 7fa0cca78aa4498e9ba57b22880c9573 [term 1 LEADER]: Becoming Leader. State: Replica: 7fa0cca78aa4498e9ba57b22880c9573, State: Running, Role: LEADER
I20240502 07:59:24.306116 17759 ts_tablet_manager.cc:1398] T 385590337e9d443a825b8a78922cda2a P 7fa0cca78aa4498e9ba57b22880c9573: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20240502 07:59:24.306171 17761 consensus_queue.cc:237] T 385590337e9d443a825b8a78922cda2a P 7fa0cca78aa4498e9ba57b22880c9573 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fa0cca78aa4498e9ba57b22880c9573" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45373 } }
I20240502 07:59:24.306315 17759 tablet_bootstrap.cc:492] T 276002f558e54354897464ea6e718c20 P 7fa0cca78aa4498e9ba57b22880c9573: Bootstrap starting.
I20240502 07:59:24.307135 17759 tablet_bootstrap.cc:654] T 276002f558e54354897464ea6e718c20 P 7fa0cca78aa4498e9ba57b22880c9573: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:24.307962 17759 tablet_bootstrap.cc:492] T 276002f558e54354897464ea6e718c20 P 7fa0cca78aa4498e9ba57b22880c9573: No bootstrap required, opened a new log
I20240502 07:59:24.308029 17759 ts_tablet_manager.cc:1367] T 276002f558e54354897464ea6e718c20 P 7fa0cca78aa4498e9ba57b22880c9573: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 07:59:24.308523 17759 raft_consensus.cc:348] T 276002f558e54354897464ea6e718c20 P 7fa0cca78aa4498e9ba57b22880c9573 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fa0cca78aa4498e9ba57b22880c9573" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45373 } }
I20240502 07:59:24.308634 17759 raft_consensus.cc:374] T 276002f558e54354897464ea6e718c20 P 7fa0cca78aa4498e9ba57b22880c9573 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:24.308681 17759 raft_consensus.cc:729] T 276002f558e54354897464ea6e718c20 P 7fa0cca78aa4498e9ba57b22880c9573 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fa0cca78aa4498e9ba57b22880c9573, State: Initialized, Role: FOLLOWER
I20240502 07:59:24.308786 17759 consensus_queue.cc:260] T 276002f558e54354897464ea6e718c20 P 7fa0cca78aa4498e9ba57b22880c9573 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fa0cca78aa4498e9ba57b22880c9573" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45373 } }
I20240502 07:59:24.308866 17759 raft_consensus.cc:388] T 276002f558e54354897464ea6e718c20 P 7fa0cca78aa4498e9ba57b22880c9573 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 07:59:24.308904 17759 raft_consensus.cc:482] T 276002f558e54354897464ea6e718c20 P 7fa0cca78aa4498e9ba57b22880c9573 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 07:59:24.308946 17759 raft_consensus.cc:3037] T 276002f558e54354897464ea6e718c20 P 7fa0cca78aa4498e9ba57b22880c9573 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:24.308856 17138 catalog_manager.cc:5484] T 385590337e9d443a825b8a78922cda2a P 7fa0cca78aa4498e9ba57b22880c9573 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7fa0cca78aa4498e9ba57b22880c9573 (127.3.139.66). New cstate: current_term: 1 leader_uuid: "7fa0cca78aa4498e9ba57b22880c9573" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fa0cca78aa4498e9ba57b22880c9573" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45373 } health_report { overall_health: HEALTHY } } }
I20240502 07:59:24.309547 17759 raft_consensus.cc:504] T 276002f558e54354897464ea6e718c20 P 7fa0cca78aa4498e9ba57b22880c9573 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fa0cca78aa4498e9ba57b22880c9573" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45373 } }
I20240502 07:59:24.309650 17759 leader_election.cc:302] T 276002f558e54354897464ea6e718c20 P 7fa0cca78aa4498e9ba57b22880c9573 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7fa0cca78aa4498e9ba57b22880c9573; no voters:
I20240502 07:59:24.309733 17759 leader_election.cc:288] T 276002f558e54354897464ea6e718c20 P 7fa0cca78aa4498e9ba57b22880c9573 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 07:59:24.309772 17761 raft_consensus.cc:2781] T 276002f558e54354897464ea6e718c20 P 7fa0cca78aa4498e9ba57b22880c9573 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:24.309855 17761 raft_consensus.cc:686] T 276002f558e54354897464ea6e718c20 P 7fa0cca78aa4498e9ba57b22880c9573 [term 1 LEADER]: Becoming Leader. State: Replica: 7fa0cca78aa4498e9ba57b22880c9573, State: Running, Role: LEADER
I20240502 07:59:24.309957 17761 consensus_queue.cc:237] T 276002f558e54354897464ea6e718c20 P 7fa0cca78aa4498e9ba57b22880c9573 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fa0cca78aa4498e9ba57b22880c9573" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45373 } }
I20240502 07:59:24.310043 17759 ts_tablet_manager.cc:1398] T 276002f558e54354897464ea6e718c20 P 7fa0cca78aa4498e9ba57b22880c9573: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20240502 07:59:24.311640 17138 catalog_manager.cc:5484] T 276002f558e54354897464ea6e718c20 P 7fa0cca78aa4498e9ba57b22880c9573 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7fa0cca78aa4498e9ba57b22880c9573 (127.3.139.66). New cstate: current_term: 1 leader_uuid: "7fa0cca78aa4498e9ba57b22880c9573" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fa0cca78aa4498e9ba57b22880c9573" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45373 } health_report { overall_health: HEALTHY } } }
I20240502 07:59:24.464226 17138 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48990:
name: "rebalance_test_table_1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:59:24.468983 17263 tablet_service.cc:1467] Processing CreateTablet for tablet 09cbdcacea3c424493775868ed27362a (DEFAULT_TABLE table=rebalance_test_table_1 [id=650a9997318a4164ba6852086bd8f5ee]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:24.469038 17394 tablet_service.cc:1467] Processing CreateTablet for tablet d592eeae00054794becc66f21977cd19 (DEFAULT_TABLE table=rebalance_test_table_1 [id=650a9997318a4164ba6852086bd8f5ee]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:24.469257 17263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09cbdcacea3c424493775868ed27362a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:24.469274 17394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d592eeae00054794becc66f21977cd19. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:24.469228 17262 tablet_service.cc:1467] Processing CreateTablet for tablet c90ce897261c4e42a3b0f27bfdad936e (DEFAULT_TABLE table=rebalance_test_table_1 [id=650a9997318a4164ba6852086bd8f5ee]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:24.469429 17262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c90ce897261c4e42a3b0f27bfdad936e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:24.470829 17759 tablet_bootstrap.cc:492] T d592eeae00054794becc66f21977cd19 P 7fa0cca78aa4498e9ba57b22880c9573: Bootstrap starting.
I20240502 07:59:24.471380 17752 tablet_bootstrap.cc:492] T c90ce897261c4e42a3b0f27bfdad936e P 428e758256a24cd78560bac584e16ab3: Bootstrap starting.
I20240502 07:59:24.471717 17759 tablet_bootstrap.cc:654] T d592eeae00054794becc66f21977cd19 P 7fa0cca78aa4498e9ba57b22880c9573: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:24.472213 17752 tablet_bootstrap.cc:654] T c90ce897261c4e42a3b0f27bfdad936e P 428e758256a24cd78560bac584e16ab3: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:24.472923 17759 tablet_bootstrap.cc:492] T d592eeae00054794becc66f21977cd19 P 7fa0cca78aa4498e9ba57b22880c9573: No bootstrap required, opened a new log
I20240502 07:59:24.472992 17759 ts_tablet_manager.cc:1367] T d592eeae00054794becc66f21977cd19 P 7fa0cca78aa4498e9ba57b22880c9573: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:24.473161 17752 tablet_bootstrap.cc:492] T c90ce897261c4e42a3b0f27bfdad936e P 428e758256a24cd78560bac584e16ab3: No bootstrap required, opened a new log
I20240502 07:59:24.473237 17752 ts_tablet_manager.cc:1367] T c90ce897261c4e42a3b0f27bfdad936e P 428e758256a24cd78560bac584e16ab3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:24.473345 17759 raft_consensus.cc:348] T d592eeae00054794becc66f21977cd19 P 7fa0cca78aa4498e9ba57b22880c9573 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fa0cca78aa4498e9ba57b22880c9573" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45373 } }
I20240502 07:59:24.473424 17759 raft_consensus.cc:374] T d592eeae00054794becc66f21977cd19 P 7fa0cca78aa4498e9ba57b22880c9573 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:24.473445 17759 raft_consensus.cc:729] T d592eeae00054794becc66f21977cd19 P 7fa0cca78aa4498e9ba57b22880c9573 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fa0cca78aa4498e9ba57b22880c9573, State: Initialized, Role: FOLLOWER
I20240502 07:59:24.473552 17759 consensus_queue.cc:260] T d592eeae00054794becc66f21977cd19 P 7fa0cca78aa4498e9ba57b22880c9573 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fa0cca78aa4498e9ba57b22880c9573" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45373 } }
I20240502 07:59:24.473613 17759 raft_consensus.cc:388] T d592eeae00054794becc66f21977cd19 P 7fa0cca78aa4498e9ba57b22880c9573 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 07:59:24.473642 17759 raft_consensus.cc:482] T d592eeae00054794becc66f21977cd19 P 7fa0cca78aa4498e9ba57b22880c9573 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 07:59:24.473685 17759 raft_consensus.cc:3037] T d592eeae00054794becc66f21977cd19 P 7fa0cca78aa4498e9ba57b22880c9573 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:24.473649 17752 raft_consensus.cc:348] T c90ce897261c4e42a3b0f27bfdad936e P 428e758256a24cd78560bac584e16ab3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "428e758256a24cd78560bac584e16ab3" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 32961 } }
I20240502 07:59:24.473735 17752 raft_consensus.cc:374] T c90ce897261c4e42a3b0f27bfdad936e P 428e758256a24cd78560bac584e16ab3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:24.473773 17752 raft_consensus.cc:729] T c90ce897261c4e42a3b0f27bfdad936e P 428e758256a24cd78560bac584e16ab3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 428e758256a24cd78560bac584e16ab3, State: Initialized, Role: FOLLOWER
I20240502 07:59:24.473883 17752 consensus_queue.cc:260] T c90ce897261c4e42a3b0f27bfdad936e P 428e758256a24cd78560bac584e16ab3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "428e758256a24cd78560bac584e16ab3" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 32961 } }
I20240502 07:59:24.473958 17752 raft_consensus.cc:388] T c90ce897261c4e42a3b0f27bfdad936e P 428e758256a24cd78560bac584e16ab3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 07:59:24.473989 17752 raft_consensus.cc:482] T c90ce897261c4e42a3b0f27bfdad936e P 428e758256a24cd78560bac584e16ab3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 07:59:24.474031 17752 raft_consensus.cc:3037] T c90ce897261c4e42a3b0f27bfdad936e P 428e758256a24cd78560bac584e16ab3 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:24.474370 17759 raft_consensus.cc:504] T d592eeae00054794becc66f21977cd19 P 7fa0cca78aa4498e9ba57b22880c9573 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fa0cca78aa4498e9ba57b22880c9573" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45373 } }
I20240502 07:59:24.474499 17759 leader_election.cc:302] T d592eeae00054794becc66f21977cd19 P 7fa0cca78aa4498e9ba57b22880c9573 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7fa0cca78aa4498e9ba57b22880c9573; no voters:
I20240502 07:59:24.474591 17759 leader_election.cc:288] T d592eeae00054794becc66f21977cd19 P 7fa0cca78aa4498e9ba57b22880c9573 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 07:59:24.474678 17752 raft_consensus.cc:504] T c90ce897261c4e42a3b0f27bfdad936e P 428e758256a24cd78560bac584e16ab3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "428e758256a24cd78560bac584e16ab3" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 32961 } }
I20240502 07:59:24.474720 17761 raft_consensus.cc:2781] T d592eeae00054794becc66f21977cd19 P 7fa0cca78aa4498e9ba57b22880c9573 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:24.474807 17761 raft_consensus.cc:686] T d592eeae00054794becc66f21977cd19 P 7fa0cca78aa4498e9ba57b22880c9573 [term 1 LEADER]: Becoming Leader. State: Replica: 7fa0cca78aa4498e9ba57b22880c9573, State: Running, Role: LEADER
I20240502 07:59:24.474803 17752 leader_election.cc:302] T c90ce897261c4e42a3b0f27bfdad936e P 428e758256a24cd78560bac584e16ab3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 428e758256a24cd78560bac584e16ab3; no voters:
I20240502 07:59:24.474936 17759 ts_tablet_manager.cc:1398] T d592eeae00054794becc66f21977cd19 P 7fa0cca78aa4498e9ba57b22880c9573: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20240502 07:59:24.474918 17761 consensus_queue.cc:237] T d592eeae00054794becc66f21977cd19 P 7fa0cca78aa4498e9ba57b22880c9573 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fa0cca78aa4498e9ba57b22880c9573" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45373 } }
I20240502 07:59:24.475343 17752 leader_election.cc:288] T c90ce897261c4e42a3b0f27bfdad936e P 428e758256a24cd78560bac584e16ab3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 07:59:24.475430 17756 raft_consensus.cc:2781] T c90ce897261c4e42a3b0f27bfdad936e P 428e758256a24cd78560bac584e16ab3 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:24.475520 17756 raft_consensus.cc:686] T c90ce897261c4e42a3b0f27bfdad936e P 428e758256a24cd78560bac584e16ab3 [term 1 LEADER]: Becoming Leader. State: Replica: 428e758256a24cd78560bac584e16ab3, State: Running, Role: LEADER
I20240502 07:59:24.475626 17756 consensus_queue.cc:237] T c90ce897261c4e42a3b0f27bfdad936e P 428e758256a24cd78560bac584e16ab3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "428e758256a24cd78560bac584e16ab3" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 32961 } }
I20240502 07:59:24.476176 17752 ts_tablet_manager.cc:1398] T c90ce897261c4e42a3b0f27bfdad936e P 428e758256a24cd78560bac584e16ab3: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:24.476296 17138 catalog_manager.cc:5484] T d592eeae00054794becc66f21977cd19 P 7fa0cca78aa4498e9ba57b22880c9573 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7fa0cca78aa4498e9ba57b22880c9573 (127.3.139.66). New cstate: current_term: 1 leader_uuid: "7fa0cca78aa4498e9ba57b22880c9573" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fa0cca78aa4498e9ba57b22880c9573" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45373 } health_report { overall_health: HEALTHY } } }
I20240502 07:59:24.476575 17752 tablet_bootstrap.cc:492] T 09cbdcacea3c424493775868ed27362a P 428e758256a24cd78560bac584e16ab3: Bootstrap starting.
I20240502 07:59:24.476915 17141 catalog_manager.cc:5484] T c90ce897261c4e42a3b0f27bfdad936e P 428e758256a24cd78560bac584e16ab3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 428e758256a24cd78560bac584e16ab3 (127.3.139.65). New cstate: current_term: 1 leader_uuid: "428e758256a24cd78560bac584e16ab3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "428e758256a24cd78560bac584e16ab3" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 32961 } health_report { overall_health: HEALTHY } } }
I20240502 07:59:24.478653 17752 tablet_bootstrap.cc:654] T 09cbdcacea3c424493775868ed27362a P 428e758256a24cd78560bac584e16ab3: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:24.479773 17752 tablet_bootstrap.cc:492] T 09cbdcacea3c424493775868ed27362a P 428e758256a24cd78560bac584e16ab3: No bootstrap required, opened a new log
I20240502 07:59:24.479836 17752 ts_tablet_manager.cc:1367] T 09cbdcacea3c424493775868ed27362a P 428e758256a24cd78560bac584e16ab3: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:24.480160 17752 raft_consensus.cc:348] T 09cbdcacea3c424493775868ed27362a P 428e758256a24cd78560bac584e16ab3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "428e758256a24cd78560bac584e16ab3" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 32961 } }
I20240502 07:59:24.480232 17752 raft_consensus.cc:374] T 09cbdcacea3c424493775868ed27362a P 428e758256a24cd78560bac584e16ab3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:24.480258 17752 raft_consensus.cc:729] T 09cbdcacea3c424493775868ed27362a P 428e758256a24cd78560bac584e16ab3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 428e758256a24cd78560bac584e16ab3, State: Initialized, Role: FOLLOWER
I20240502 07:59:24.480366 17752 consensus_queue.cc:260] T 09cbdcacea3c424493775868ed27362a P 428e758256a24cd78560bac584e16ab3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "428e758256a24cd78560bac584e16ab3" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 32961 } }
I20240502 07:59:24.480484 17752 raft_consensus.cc:388] T 09cbdcacea3c424493775868ed27362a P 428e758256a24cd78560bac584e16ab3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 07:59:24.480526 17752 raft_consensus.cc:482] T 09cbdcacea3c424493775868ed27362a P 428e758256a24cd78560bac584e16ab3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 07:59:24.480573 17752 raft_consensus.cc:3037] T 09cbdcacea3c424493775868ed27362a P 428e758256a24cd78560bac584e16ab3 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:24.481494 17752 raft_consensus.cc:504] T 09cbdcacea3c424493775868ed27362a P 428e758256a24cd78560bac584e16ab3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "428e758256a24cd78560bac584e16ab3" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 32961 } }
I20240502 07:59:24.481590 17752 leader_election.cc:302] T 09cbdcacea3c424493775868ed27362a P 428e758256a24cd78560bac584e16ab3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 428e758256a24cd78560bac584e16ab3; no voters:
I20240502 07:59:24.481649 17752 leader_election.cc:288] T 09cbdcacea3c424493775868ed27362a P 428e758256a24cd78560bac584e16ab3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 07:59:24.481698 17756 raft_consensus.cc:2781] T 09cbdcacea3c424493775868ed27362a P 428e758256a24cd78560bac584e16ab3 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:24.481806 17756 raft_consensus.cc:686] T 09cbdcacea3c424493775868ed27362a P 428e758256a24cd78560bac584e16ab3 [term 1 LEADER]: Becoming Leader. State: Replica: 428e758256a24cd78560bac584e16ab3, State: Running, Role: LEADER
I20240502 07:59:24.481827 17752 ts_tablet_manager.cc:1398] T 09cbdcacea3c424493775868ed27362a P 428e758256a24cd78560bac584e16ab3: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:24.481928 17756 consensus_queue.cc:237] T 09cbdcacea3c424493775868ed27362a P 428e758256a24cd78560bac584e16ab3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "428e758256a24cd78560bac584e16ab3" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 32961 } }
I20240502 07:59:24.482962 17141 catalog_manager.cc:5484] T 09cbdcacea3c424493775868ed27362a P 428e758256a24cd78560bac584e16ab3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 428e758256a24cd78560bac584e16ab3 (127.3.139.65). New cstate: current_term: 1 leader_uuid: "428e758256a24cd78560bac584e16ab3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "428e758256a24cd78560bac584e16ab3" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 32961 } health_report { overall_health: HEALTHY } } }
I20240502 07:59:24.632666 17138 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48990:
name: "rebalance_test_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:59:24.637388 17393 tablet_service.cc:1467] Processing CreateTablet for tablet f4255a612f6247ee83cc6ac2c07c4c18 (DEFAULT_TABLE table=rebalance_test_table_2 [id=1fe327d59c8348cab65cfaf105789006]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:24.637388 17394 tablet_service.cc:1467] Processing CreateTablet for tablet d55bcea3e0e54d3990f494d3ccb3bc68 (DEFAULT_TABLE table=rebalance_test_table_2 [id=1fe327d59c8348cab65cfaf105789006]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:24.637554 17263 tablet_service.cc:1467] Processing CreateTablet for tablet f7948b810fe449a894e3d572c0aacf44 (DEFAULT_TABLE table=rebalance_test_table_2 [id=1fe327d59c8348cab65cfaf105789006]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:24.637702 17393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4255a612f6247ee83cc6ac2c07c4c18. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:24.637765 17263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7948b810fe449a894e3d572c0aacf44. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:24.637816 17394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d55bcea3e0e54d3990f494d3ccb3bc68. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:24.639370 17759 tablet_bootstrap.cc:492] T d55bcea3e0e54d3990f494d3ccb3bc68 P 7fa0cca78aa4498e9ba57b22880c9573: Bootstrap starting.
I20240502 07:59:24.639565 17752 tablet_bootstrap.cc:492] T f7948b810fe449a894e3d572c0aacf44 P 428e758256a24cd78560bac584e16ab3: Bootstrap starting.
I20240502 07:59:24.640275 17759 tablet_bootstrap.cc:654] T d55bcea3e0e54d3990f494d3ccb3bc68 P 7fa0cca78aa4498e9ba57b22880c9573: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:24.640364 17752 tablet_bootstrap.cc:654] T f7948b810fe449a894e3d572c0aacf44 P 428e758256a24cd78560bac584e16ab3: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:24.641196 17759 tablet_bootstrap.cc:492] T d55bcea3e0e54d3990f494d3ccb3bc68 P 7fa0cca78aa4498e9ba57b22880c9573: No bootstrap required, opened a new log
I20240502 07:59:24.641202 17752 tablet_bootstrap.cc:492] T f7948b810fe449a894e3d572c0aacf44 P 428e758256a24cd78560bac584e16ab3: No bootstrap required, opened a new log
I20240502 07:59:24.641264 17759 ts_tablet_manager.cc:1367] T d55bcea3e0e54d3990f494d3ccb3bc68 P 7fa0cca78aa4498e9ba57b22880c9573: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:24.641266 17752 ts_tablet_manager.cc:1367] T f7948b810fe449a894e3d572c0aacf44 P 428e758256a24cd78560bac584e16ab3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:24.641602 17752 raft_consensus.cc:348] T f7948b810fe449a894e3d572c0aacf44 P 428e758256a24cd78560bac584e16ab3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "428e758256a24cd78560bac584e16ab3" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 32961 } }
I20240502 07:59:24.641611 17759 raft_consensus.cc:348] T d55bcea3e0e54d3990f494d3ccb3bc68 P 7fa0cca78aa4498e9ba57b22880c9573 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fa0cca78aa4498e9ba57b22880c9573" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45373 } }
I20240502 07:59:24.641678 17759 raft_consensus.cc:374] T d55bcea3e0e54d3990f494d3ccb3bc68 P 7fa0cca78aa4498e9ba57b22880c9573 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:24.641680 17752 raft_consensus.cc:374] T f7948b810fe449a894e3d572c0aacf44 P 428e758256a24cd78560bac584e16ab3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:24.641696 17759 raft_consensus.cc:729] T d55bcea3e0e54d3990f494d3ccb3bc68 P 7fa0cca78aa4498e9ba57b22880c9573 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fa0cca78aa4498e9ba57b22880c9573, State: Initialized, Role: FOLLOWER
I20240502 07:59:24.641707 17752 raft_consensus.cc:729] T f7948b810fe449a894e3d572c0aacf44 P 428e758256a24cd78560bac584e16ab3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 428e758256a24cd78560bac584e16ab3, State: Initialized, Role: FOLLOWER
I20240502 07:59:24.641794 17759 consensus_queue.cc:260] T d55bcea3e0e54d3990f494d3ccb3bc68 P 7fa0cca78aa4498e9ba57b22880c9573 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fa0cca78aa4498e9ba57b22880c9573" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45373 } }
I20240502 07:59:24.641860 17759 raft_consensus.cc:388] T d55bcea3e0e54d3990f494d3ccb3bc68 P 7fa0cca78aa4498e9ba57b22880c9573 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 07:59:24.641822 17752 consensus_queue.cc:260] T f7948b810fe449a894e3d572c0aacf44 P 428e758256a24cd78560bac584e16ab3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "428e758256a24cd78560bac584e16ab3" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 32961 } }
I20240502 07:59:24.641883 17759 raft_consensus.cc:482] T d55bcea3e0e54d3990f494d3ccb3bc68 P 7fa0cca78aa4498e9ba57b22880c9573 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 07:59:24.641920 17759 raft_consensus.cc:3037] T d55bcea3e0e54d3990f494d3ccb3bc68 P 7fa0cca78aa4498e9ba57b22880c9573 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:24.641922 17752 raft_consensus.cc:388] T f7948b810fe449a894e3d572c0aacf44 P 428e758256a24cd78560bac584e16ab3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 07:59:24.641955 17752 raft_consensus.cc:482] T f7948b810fe449a894e3d572c0aacf44 P 428e758256a24cd78560bac584e16ab3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 07:59:24.641999 17752 raft_consensus.cc:3037] T f7948b810fe449a894e3d572c0aacf44 P 428e758256a24cd78560bac584e16ab3 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:24.642520 17759 raft_consensus.cc:504] T d55bcea3e0e54d3990f494d3ccb3bc68 P 7fa0cca78aa4498e9ba57b22880c9573 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fa0cca78aa4498e9ba57b22880c9573" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45373 } }
I20240502 07:59:24.642568 17752 raft_consensus.cc:504] T f7948b810fe449a894e3d572c0aacf44 P 428e758256a24cd78560bac584e16ab3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "428e758256a24cd78560bac584e16ab3" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 32961 } }
I20240502 07:59:24.642635 17759 leader_election.cc:302] T d55bcea3e0e54d3990f494d3ccb3bc68 P 7fa0cca78aa4498e9ba57b22880c9573 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7fa0cca78aa4498e9ba57b22880c9573; no voters:
I20240502 07:59:24.642664 17752 leader_election.cc:302] T f7948b810fe449a894e3d572c0aacf44 P 428e758256a24cd78560bac584e16ab3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 428e758256a24cd78560bac584e16ab3; no voters:
I20240502 07:59:24.642724 17759 leader_election.cc:288] T d55bcea3e0e54d3990f494d3ccb3bc68 P 7fa0cca78aa4498e9ba57b22880c9573 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 07:59:24.642771 17761 raft_consensus.cc:2781] T d55bcea3e0e54d3990f494d3ccb3bc68 P 7fa0cca78aa4498e9ba57b22880c9573 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:24.642855 17761 raft_consensus.cc:686] T d55bcea3e0e54d3990f494d3ccb3bc68 P 7fa0cca78aa4498e9ba57b22880c9573 [term 1 LEADER]: Becoming Leader. State: Replica: 7fa0cca78aa4498e9ba57b22880c9573, State: Running, Role: LEADER
I20240502 07:59:24.642962 17761 consensus_queue.cc:237] T d55bcea3e0e54d3990f494d3ccb3bc68 P 7fa0cca78aa4498e9ba57b22880c9573 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fa0cca78aa4498e9ba57b22880c9573" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45373 } }
I20240502 07:59:24.643224 17759 ts_tablet_manager.cc:1398] T d55bcea3e0e54d3990f494d3ccb3bc68 P 7fa0cca78aa4498e9ba57b22880c9573: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:24.643404 17752 leader_election.cc:288] T f7948b810fe449a894e3d572c0aacf44 P 428e758256a24cd78560bac584e16ab3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 07:59:24.643445 17756 raft_consensus.cc:2781] T f7948b810fe449a894e3d572c0aacf44 P 428e758256a24cd78560bac584e16ab3 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:24.643682 17752 ts_tablet_manager.cc:1398] T f7948b810fe449a894e3d572c0aacf44 P 428e758256a24cd78560bac584e16ab3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:24.643723 17759 tablet_bootstrap.cc:492] T f4255a612f6247ee83cc6ac2c07c4c18 P 7fa0cca78aa4498e9ba57b22880c9573: Bootstrap starting.
I20240502 07:59:24.643934 17756 raft_consensus.cc:686] T f7948b810fe449a894e3d572c0aacf44 P 428e758256a24cd78560bac584e16ab3 [term 1 LEADER]: Becoming Leader. State: Replica: 428e758256a24cd78560bac584e16ab3, State: Running, Role: LEADER
I20240502 07:59:24.644191 17756 consensus_queue.cc:237] T f7948b810fe449a894e3d572c0aacf44 P 428e758256a24cd78560bac584e16ab3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "428e758256a24cd78560bac584e16ab3" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 32961 } }
I20240502 07:59:24.644289 17138 catalog_manager.cc:5484] T d55bcea3e0e54d3990f494d3ccb3bc68 P 7fa0cca78aa4498e9ba57b22880c9573 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7fa0cca78aa4498e9ba57b22880c9573 (127.3.139.66). New cstate: current_term: 1 leader_uuid: "7fa0cca78aa4498e9ba57b22880c9573" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fa0cca78aa4498e9ba57b22880c9573" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45373 } health_report { overall_health: HEALTHY } } }
I20240502 07:59:24.644578 17759 tablet_bootstrap.cc:654] T f4255a612f6247ee83cc6ac2c07c4c18 P 7fa0cca78aa4498e9ba57b22880c9573: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:24.645460 17141 catalog_manager.cc:5484] T f7948b810fe449a894e3d572c0aacf44 P 428e758256a24cd78560bac584e16ab3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 428e758256a24cd78560bac584e16ab3 (127.3.139.65). New cstate: current_term: 1 leader_uuid: "428e758256a24cd78560bac584e16ab3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "428e758256a24cd78560bac584e16ab3" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 32961 } health_report { overall_health: HEALTHY } } }
I20240502 07:59:24.646476 17759 tablet_bootstrap.cc:492] T f4255a612f6247ee83cc6ac2c07c4c18 P 7fa0cca78aa4498e9ba57b22880c9573: No bootstrap required, opened a new log
I20240502 07:59:24.646553 17759 ts_tablet_manager.cc:1367] T f4255a612f6247ee83cc6ac2c07c4c18 P 7fa0cca78aa4498e9ba57b22880c9573: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:59:24.646986 17759 raft_consensus.cc:348] T f4255a612f6247ee83cc6ac2c07c4c18 P 7fa0cca78aa4498e9ba57b22880c9573 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fa0cca78aa4498e9ba57b22880c9573" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45373 } }
I20240502 07:59:24.647089 17759 raft_consensus.cc:374] T f4255a612f6247ee83cc6ac2c07c4c18 P 7fa0cca78aa4498e9ba57b22880c9573 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:24.647120 17759 raft_consensus.cc:729] T f4255a612f6247ee83cc6ac2c07c4c18 P 7fa0cca78aa4498e9ba57b22880c9573 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fa0cca78aa4498e9ba57b22880c9573, State: Initialized, Role: FOLLOWER
I20240502 07:59:24.647217 17759 consensus_queue.cc:260] T f4255a612f6247ee83cc6ac2c07c4c18 P 7fa0cca78aa4498e9ba57b22880c9573 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fa0cca78aa4498e9ba57b22880c9573" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45373 } }
I20240502 07:59:24.647287 17759 raft_consensus.cc:388] T f4255a612f6247ee83cc6ac2c07c4c18 P 7fa0cca78aa4498e9ba57b22880c9573 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 07:59:24.647320 17759 raft_consensus.cc:482] T f4255a612f6247ee83cc6ac2c07c4c18 P 7fa0cca78aa4498e9ba57b22880c9573 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 07:59:24.647359 17759 raft_consensus.cc:3037] T f4255a612f6247ee83cc6ac2c07c4c18 P 7fa0cca78aa4498e9ba57b22880c9573 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:24.648085 17759 raft_consensus.cc:504] T f4255a612f6247ee83cc6ac2c07c4c18 P 7fa0cca78aa4498e9ba57b22880c9573 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fa0cca78aa4498e9ba57b22880c9573" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45373 } }
I20240502 07:59:24.648203 17759 leader_election.cc:302] T f4255a612f6247ee83cc6ac2c07c4c18 P 7fa0cca78aa4498e9ba57b22880c9573 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7fa0cca78aa4498e9ba57b22880c9573; no voters:
I20240502 07:59:24.648289 17759 leader_election.cc:288] T f4255a612f6247ee83cc6ac2c07c4c18 P 7fa0cca78aa4498e9ba57b22880c9573 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 07:59:24.648348 17762 raft_consensus.cc:2781] T f4255a612f6247ee83cc6ac2c07c4c18 P 7fa0cca78aa4498e9ba57b22880c9573 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:24.648425 17762 raft_consensus.cc:686] T f4255a612f6247ee83cc6ac2c07c4c18 P 7fa0cca78aa4498e9ba57b22880c9573 [term 1 LEADER]: Becoming Leader. State: Replica: 7fa0cca78aa4498e9ba57b22880c9573, State: Running, Role: LEADER
I20240502 07:59:24.648476 17759 ts_tablet_manager.cc:1398] T f4255a612f6247ee83cc6ac2c07c4c18 P 7fa0cca78aa4498e9ba57b22880c9573: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:24.648535 17762 consensus_queue.cc:237] T f4255a612f6247ee83cc6ac2c07c4c18 P 7fa0cca78aa4498e9ba57b22880c9573 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fa0cca78aa4498e9ba57b22880c9573" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45373 } }
I20240502 07:59:24.649634 17138 catalog_manager.cc:5484] T f4255a612f6247ee83cc6ac2c07c4c18 P 7fa0cca78aa4498e9ba57b22880c9573 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7fa0cca78aa4498e9ba57b22880c9573 (127.3.139.66). New cstate: current_term: 1 leader_uuid: "7fa0cca78aa4498e9ba57b22880c9573" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fa0cca78aa4498e9ba57b22880c9573" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45373 } health_report { overall_health: HEALTHY } } }
I20240502 07:59:24.799736 17138 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48990:
name: "rebalance_test_table_3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:59:24.804425 17263 tablet_service.cc:1467] Processing CreateTablet for tablet 126caa6890ff469c91a4fa20d211ed0d (DEFAULT_TABLE table=rebalance_test_table_3 [id=f8d90d315bf94e019631a1dbf9a453e8]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:24.804455 17393 tablet_service.cc:1467] Processing CreateTablet for tablet 05efac4ade434a498c56c44a86f9ccdb (DEFAULT_TABLE table=rebalance_test_table_3 [id=f8d90d315bf94e019631a1dbf9a453e8]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:24.804716 17263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 126caa6890ff469c91a4fa20d211ed0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:24.804718 17393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05efac4ade434a498c56c44a86f9ccdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:24.805171 17262 tablet_service.cc:1467] Processing CreateTablet for tablet 8327ae04d10c4caaad35984d10c79ffe (DEFAULT_TABLE table=rebalance_test_table_3 [id=f8d90d315bf94e019631a1dbf9a453e8]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:24.805368 17262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8327ae04d10c4caaad35984d10c79ffe. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:24.806399 17759 tablet_bootstrap.cc:492] T 05efac4ade434a498c56c44a86f9ccdb P 7fa0cca78aa4498e9ba57b22880c9573: Bootstrap starting.
I20240502 07:59:24.806563 17752 tablet_bootstrap.cc:492] T 126caa6890ff469c91a4fa20d211ed0d P 428e758256a24cd78560bac584e16ab3: Bootstrap starting.
I20240502 07:59:24.807454 17759 tablet_bootstrap.cc:654] T 05efac4ade434a498c56c44a86f9ccdb P 7fa0cca78aa4498e9ba57b22880c9573: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:24.807456 17752 tablet_bootstrap.cc:654] T 126caa6890ff469c91a4fa20d211ed0d P 428e758256a24cd78560bac584e16ab3: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:24.808450 17759 tablet_bootstrap.cc:492] T 05efac4ade434a498c56c44a86f9ccdb P 7fa0cca78aa4498e9ba57b22880c9573: No bootstrap required, opened a new log
I20240502 07:59:24.808526 17759 ts_tablet_manager.cc:1367] T 05efac4ade434a498c56c44a86f9ccdb P 7fa0cca78aa4498e9ba57b22880c9573: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 07:59:24.808670 17752 tablet_bootstrap.cc:492] T 126caa6890ff469c91a4fa20d211ed0d P 428e758256a24cd78560bac584e16ab3: No bootstrap required, opened a new log
I20240502 07:59:24.808743 17752 ts_tablet_manager.cc:1367] T 126caa6890ff469c91a4fa20d211ed0d P 428e758256a24cd78560bac584e16ab3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:24.808926 17759 raft_consensus.cc:348] T 05efac4ade434a498c56c44a86f9ccdb P 7fa0cca78aa4498e9ba57b22880c9573 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fa0cca78aa4498e9ba57b22880c9573" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45373 } }
I20240502 07:59:24.808993 17759 raft_consensus.cc:374] T 05efac4ade434a498c56c44a86f9ccdb P 7fa0cca78aa4498e9ba57b22880c9573 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:24.809012 17759 raft_consensus.cc:729] T 05efac4ade434a498c56c44a86f9ccdb P 7fa0cca78aa4498e9ba57b22880c9573 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fa0cca78aa4498e9ba57b22880c9573, State: Initialized, Role: FOLLOWER
I20240502 07:59:24.809099 17759 consensus_queue.cc:260] T 05efac4ade434a498c56c44a86f9ccdb P 7fa0cca78aa4498e9ba57b22880c9573 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fa0cca78aa4498e9ba57b22880c9573" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45373 } }
I20240502 07:59:24.809084 17752 raft_consensus.cc:348] T 126caa6890ff469c91a4fa20d211ed0d P 428e758256a24cd78560bac584e16ab3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "428e758256a24cd78560bac584e16ab3" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 32961 } }
I20240502 07:59:24.809145 17759 raft_consensus.cc:388] T 05efac4ade434a498c56c44a86f9ccdb P 7fa0cca78aa4498e9ba57b22880c9573 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 07:59:24.809159 17752 raft_consensus.cc:374] T 126caa6890ff469c91a4fa20d211ed0d P 428e758256a24cd78560bac584e16ab3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:24.809165 17759 raft_consensus.cc:482] T 05efac4ade434a498c56c44a86f9ccdb P 7fa0cca78aa4498e9ba57b22880c9573 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 07:59:24.809180 17752 raft_consensus.cc:729] T 126caa6890ff469c91a4fa20d211ed0d P 428e758256a24cd78560bac584e16ab3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 428e758256a24cd78560bac584e16ab3, State: Initialized, Role: FOLLOWER
I20240502 07:59:24.809191 17759 raft_consensus.cc:3037] T 05efac4ade434a498c56c44a86f9ccdb P 7fa0cca78aa4498e9ba57b22880c9573 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:24.809265 17752 consensus_queue.cc:260] T 126caa6890ff469c91a4fa20d211ed0d P 428e758256a24cd78560bac584e16ab3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "428e758256a24cd78560bac584e16ab3" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 32961 } }
I20240502 07:59:24.809314 17752 raft_consensus.cc:388] T 126caa6890ff469c91a4fa20d211ed0d P 428e758256a24cd78560bac584e16ab3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 07:59:24.809332 17752 raft_consensus.cc:482] T 126caa6890ff469c91a4fa20d211ed0d P 428e758256a24cd78560bac584e16ab3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 07:59:24.809360 17752 raft_consensus.cc:3037] T 126caa6890ff469c91a4fa20d211ed0d P 428e758256a24cd78560bac584e16ab3 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:24.809756 17759 raft_consensus.cc:504] T 05efac4ade434a498c56c44a86f9ccdb P 7fa0cca78aa4498e9ba57b22880c9573 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fa0cca78aa4498e9ba57b22880c9573" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45373 } }
I20240502 07:59:24.809851 17759 leader_election.cc:302] T 05efac4ade434a498c56c44a86f9ccdb P 7fa0cca78aa4498e9ba57b22880c9573 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7fa0cca78aa4498e9ba57b22880c9573; no voters:
I20240502 07:59:24.809926 17759 leader_election.cc:288] T 05efac4ade434a498c56c44a86f9ccdb P 7fa0cca78aa4498e9ba57b22880c9573 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 07:59:24.809890 17752 raft_consensus.cc:504] T 126caa6890ff469c91a4fa20d211ed0d P 428e758256a24cd78560bac584e16ab3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "428e758256a24cd78560bac584e16ab3" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 32961 } }
I20240502 07:59:24.809998 17762 raft_consensus.cc:2781] T 05efac4ade434a498c56c44a86f9ccdb P 7fa0cca78aa4498e9ba57b22880c9573 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:24.810004 17752 leader_election.cc:302] T 126caa6890ff469c91a4fa20d211ed0d P 428e758256a24cd78560bac584e16ab3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 428e758256a24cd78560bac584e16ab3; no voters:
I20240502 07:59:24.810077 17752 leader_election.cc:288] T 126caa6890ff469c91a4fa20d211ed0d P 428e758256a24cd78560bac584e16ab3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 07:59:24.810104 17762 raft_consensus.cc:686] T 05efac4ade434a498c56c44a86f9ccdb P 7fa0cca78aa4498e9ba57b22880c9573 [term 1 LEADER]: Becoming Leader. State: Replica: 7fa0cca78aa4498e9ba57b22880c9573, State: Running, Role: LEADER
I20240502 07:59:24.810118 17756 raft_consensus.cc:2781] T 126caa6890ff469c91a4fa20d211ed0d P 428e758256a24cd78560bac584e16ab3 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:24.810180 17756 raft_consensus.cc:686] T 126caa6890ff469c91a4fa20d211ed0d P 428e758256a24cd78560bac584e16ab3 [term 1 LEADER]: Becoming Leader. State: Replica: 428e758256a24cd78560bac584e16ab3, State: Running, Role: LEADER
I20240502 07:59:24.810220 17762 consensus_queue.cc:237] T 05efac4ade434a498c56c44a86f9ccdb P 7fa0cca78aa4498e9ba57b22880c9573 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fa0cca78aa4498e9ba57b22880c9573" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45373 } }
I20240502 07:59:24.810276 17756 consensus_queue.cc:237] T 126caa6890ff469c91a4fa20d211ed0d P 428e758256a24cd78560bac584e16ab3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "428e758256a24cd78560bac584e16ab3" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 32961 } }
I20240502 07:59:24.810310 17759 ts_tablet_manager.cc:1398] T 05efac4ade434a498c56c44a86f9ccdb P 7fa0cca78aa4498e9ba57b22880c9573: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20240502 07:59:24.810495 17752 ts_tablet_manager.cc:1398] T 126caa6890ff469c91a4fa20d211ed0d P 428e758256a24cd78560bac584e16ab3: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:24.810628 17752 tablet_bootstrap.cc:492] T 8327ae04d10c4caaad35984d10c79ffe P 428e758256a24cd78560bac584e16ab3: Bootstrap starting.
I20240502 07:59:24.811640 17141 catalog_manager.cc:5484] T 126caa6890ff469c91a4fa20d211ed0d P 428e758256a24cd78560bac584e16ab3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 428e758256a24cd78560bac584e16ab3 (127.3.139.65). New cstate: current_term: 1 leader_uuid: "428e758256a24cd78560bac584e16ab3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "428e758256a24cd78560bac584e16ab3" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 32961 } health_report { overall_health: HEALTHY } } }
I20240502 07:59:24.811744 17138 catalog_manager.cc:5484] T 05efac4ade434a498c56c44a86f9ccdb P 7fa0cca78aa4498e9ba57b22880c9573 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7fa0cca78aa4498e9ba57b22880c9573 (127.3.139.66). New cstate: current_term: 1 leader_uuid: "7fa0cca78aa4498e9ba57b22880c9573" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fa0cca78aa4498e9ba57b22880c9573" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45373 } health_report { overall_health: HEALTHY } } }
I20240502 07:59:24.812383 17752 tablet_bootstrap.cc:654] T 8327ae04d10c4caaad35984d10c79ffe P 428e758256a24cd78560bac584e16ab3: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:24.813832 17752 tablet_bootstrap.cc:492] T 8327ae04d10c4caaad35984d10c79ffe P 428e758256a24cd78560bac584e16ab3: No bootstrap required, opened a new log
I20240502 07:59:24.813896 17752 ts_tablet_manager.cc:1367] T 8327ae04d10c4caaad35984d10c79ffe P 428e758256a24cd78560bac584e16ab3: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:59:24.814225 17752 raft_consensus.cc:348] T 8327ae04d10c4caaad35984d10c79ffe P 428e758256a24cd78560bac584e16ab3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "428e758256a24cd78560bac584e16ab3" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 32961 } }
I20240502 07:59:24.814307 17752 raft_consensus.cc:374] T 8327ae04d10c4caaad35984d10c79ffe P 428e758256a24cd78560bac584e16ab3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:24.814337 17752 raft_consensus.cc:729] T 8327ae04d10c4caaad35984d10c79ffe P 428e758256a24cd78560bac584e16ab3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 428e758256a24cd78560bac584e16ab3, State: Initialized, Role: FOLLOWER
I20240502 07:59:24.814441 17752 consensus_queue.cc:260] T 8327ae04d10c4caaad35984d10c79ffe P 428e758256a24cd78560bac584e16ab3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "428e758256a24cd78560bac584e16ab3" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 32961 } }
I20240502 07:59:24.814512 17752 raft_consensus.cc:388] T 8327ae04d10c4caaad35984d10c79ffe P 428e758256a24cd78560bac584e16ab3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 07:59:24.814541 17752 raft_consensus.cc:482] T 8327ae04d10c4caaad35984d10c79ffe P 428e758256a24cd78560bac584e16ab3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 07:59:24.814574 17752 raft_consensus.cc:3037] T 8327ae04d10c4caaad35984d10c79ffe P 428e758256a24cd78560bac584e16ab3 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:24.815272 17752 raft_consensus.cc:504] T 8327ae04d10c4caaad35984d10c79ffe P 428e758256a24cd78560bac584e16ab3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "428e758256a24cd78560bac584e16ab3" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 32961 } }
I20240502 07:59:24.815371 17752 leader_election.cc:302] T 8327ae04d10c4caaad35984d10c79ffe P 428e758256a24cd78560bac584e16ab3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 428e758256a24cd78560bac584e16ab3; no voters:
I20240502 07:59:24.815448 17752 leader_election.cc:288] T 8327ae04d10c4caaad35984d10c79ffe P 428e758256a24cd78560bac584e16ab3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 07:59:24.815495 17756 raft_consensus.cc:2781] T 8327ae04d10c4caaad35984d10c79ffe P 428e758256a24cd78560bac584e16ab3 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:24.815575 17756 raft_consensus.cc:686] T 8327ae04d10c4caaad35984d10c79ffe P 428e758256a24cd78560bac584e16ab3 [term 1 LEADER]: Becoming Leader. State: Replica: 428e758256a24cd78560bac584e16ab3, State: Running, Role: LEADER
I20240502 07:59:24.815636 17752 ts_tablet_manager.cc:1398] T 8327ae04d10c4caaad35984d10c79ffe P 428e758256a24cd78560bac584e16ab3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:24.815668 17756 consensus_queue.cc:237] T 8327ae04d10c4caaad35984d10c79ffe P 428e758256a24cd78560bac584e16ab3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "428e758256a24cd78560bac584e16ab3" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 32961 } }
I20240502 07:59:24.817142 17138 catalog_manager.cc:5484] T 8327ae04d10c4caaad35984d10c79ffe P 428e758256a24cd78560bac584e16ab3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 428e758256a24cd78560bac584e16ab3 (127.3.139.65). New cstate: current_term: 1 leader_uuid: "428e758256a24cd78560bac584e16ab3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "428e758256a24cd78560bac584e16ab3" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 32961 } health_report { overall_health: HEALTHY } } }
I20240502 07:59:24.962565 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.67:36601
--local_ip_for_outbound_sockets=127.3.139.67
--tserver_master_addrs=127.3.139.126:41843
--webserver_port=41581
--webserver_interface=127.3.139.67
--builtin_ntp_servers=127.3.139.84:44509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240502 07:59:25.059621 17858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:25.059789 17858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:59:25.059832 17858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:25.062705 17858 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:59:25.062759 17858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:59:25.062834 17858 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.67
I20240502 07:59:25.066223 17858 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:44509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.67:36601
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.3.139.67
--webserver_port=41581
--tserver_master_addrs=127.3.139.126:41843
--never_fsync=true
--heap_profile_path=/tmp/kudu.17858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.67
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:59:25.066992 17858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:59:25.067993 17858 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:59:25.075526 17863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:25.075898 17866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:25.076272 17864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:25.076704 17858 server_base.cc:1034] running on GCE node
I20240502 07:59:25.077033 17858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:59:25.077522 17858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:59:25.078668 17858 hybrid_clock.cc:648] HybridClock initialized: now 1714636765078657 us; error 25 us; skew 500 ppm
I20240502 07:59:25.080582 17858 webserver.cc:460] Webserver started at http://127.3.139.67:41581/ using document root <none> and password file <none>
I20240502 07:59:25.081069 17858 fs_manager.cc:359] Metadata directory not provided
I20240502 07:59:25.081135 17858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:59:25.084157 17858 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240502 07:59:25.085798 17872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:25.086669 17858 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:25.086755 17858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "b9991ebec9ab49ee88c8dd2cacfd6c56"
format_stamp: "Formatted at 2024-05-02 07:59:20 on dist-test-slave-twcr"
I20240502 07:59:25.087152 17858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:59:25.098950 17858 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:59:25.099401 17858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:59:25.099531 17858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:59:25.099994 17858 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:59:25.100731 17858 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:59:25.100781 17858 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:25.100814 17858 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:59:25.100836 17858 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:25.108336 17858 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.67:36601
I20240502 07:59:25.108379 17985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.67:36601 every 8 connection(s)
I20240502 07:59:25.109248 17858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240502 07:59:25.116865 17986 heartbeater.cc:347] Connected to a master server at 127.3.139.126:41843
I20240502 07:59:25.117063 17986 heartbeater.cc:464] Registering TS with master...
I20240502 07:59:25.117143 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 17858
I20240502 07:59:25.117522 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.68:33975
--local_ip_for_outbound_sockets=127.3.139.68
--tserver_master_addrs=127.3.139.126:41843
--webserver_port=39493
--webserver_interface=127.3.139.68
--builtin_ntp_servers=127.3.139.84:44509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 07:59:25.117568 17986 heartbeater.cc:511] Master 127.3.139.126:41843 requested a full tablet report, sending...
I20240502 07:59:25.118683 17138 ts_manager.cc:194] Re-registered known tserver with Master: b9991ebec9ab49ee88c8dd2cacfd6c56 (127.3.139.67:36601)
I20240502 07:59:25.119304 17138 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.67:42961
W20240502 07:59:25.214987 17989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:25.215168 17989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:59:25.215214 17989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:25.218058 17989 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:59:25.218111 17989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:59:25.218194 17989 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.68
I20240502 07:59:25.221546 17989 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:44509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.68:33975
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.3.139.68
--webserver_port=39493
--tserver_master_addrs=127.3.139.126:41843
--never_fsync=true
--heap_profile_path=/tmp/kudu.17989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.68
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:59:25.222231 17989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:59:25.223171 17989 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:59:25.230199 17994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:25.230518 17997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:25.230886 17995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:25.231281 17989 server_base.cc:1034] running on GCE node
I20240502 07:59:25.231582 17989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:59:25.232034 17989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:59:25.233166 17989 hybrid_clock.cc:648] HybridClock initialized: now 1714636765233158 us; error 22 us; skew 500 ppm
I20240502 07:59:25.234709 17989 webserver.cc:460] Webserver started at http://127.3.139.68:39493/ using document root <none> and password file <none>
I20240502 07:59:25.235236 17989 fs_manager.cc:359] Metadata directory not provided
I20240502 07:59:25.235304 17989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:59:25.238145 17989 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:25.239749 18003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:25.240607 17989 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:25.240701 17989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "db1de15a6fb94d6f83895fc5f1b6c809"
format_stamp: "Formatted at 2024-05-02 07:59:20 on dist-test-slave-twcr"
I20240502 07:59:25.241032 17989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:59:25.262115 17989 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:59:25.262540 17989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:59:25.262652 17989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:59:25.263039 17989 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:59:25.263787 17989 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:59:25.263835 17989 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:25.263882 17989 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:59:25.263902 17989 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:25.271348 17989 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.68:33975
I20240502 07:59:25.271389 18116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.68:33975 every 8 connection(s)
I20240502 07:59:25.272238 17989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240502 07:59:25.272507 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 17989
I20240502 07:59:25.280162 18117 heartbeater.cc:347] Connected to a master server at 127.3.139.126:41843
I20240502 07:59:25.280373 18117 heartbeater.cc:464] Registering TS with master...
I20240502 07:59:25.280959 18117 heartbeater.cc:511] Master 127.3.139.126:41843 requested a full tablet report, sending...
I20240502 07:59:25.281921 17138 ts_manager.cc:194] Re-registered known tserver with Master: db1de15a6fb94d6f83895fc5f1b6c809 (127.3.139.68:33975)
I20240502 07:59:25.282420 17138 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.68:50787
I20240502 07:59:25.443529 17393 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:25.443825 17262 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:25.446065 18051 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:25.446445 17920 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:25.543694 17393 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:25.544471 17262 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:25.544957 18051 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:25.551450 17920 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:25.627471 17262 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:25.628487 17920 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:25.629029 17393 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:25.629853 18051 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
0624d5d81dad41b68933fbeb4cf0e8dc | 127.3.139.126:41843 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.3.139.84:44509 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
428e758256a24cd78560bac584e16ab3 | 127.3.139.65:32961 | HEALTHY | <none> | 7 | 0
7fa0cca78aa4498e9ba57b22880c9573 | 127.3.139.66:45373 | HEALTHY | <none> | 6 | 0
b9991ebec9ab49ee88c8dd2cacfd6c56 | 127.3.139.67:36601 | HEALTHY | <none> | 0 | 0
db1de15a6fb94d6f83895fc5f1b6c809 | 127.3.139.68:33975 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_server_key_size | 768 | experimental | all 4 server(s) checked
local_ip_for_outbound_sockets | 127.3.139.65 | experimental | 127.3.139.65:32961
local_ip_for_outbound_sockets | 127.3.139.66 | experimental | 127.3.139.66:45373
local_ip_for_outbound_sockets | 127.3.139.67 | experimental | 127.3.139.67:36601
local_ip_for_outbound_sockets | 127.3.139.68 | experimental | 127.3.139.68:33975
never_fsync | true | unsafe,advanced | all 4 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 4 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 4 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
server_dump_info_format | pb | hidden | all 4 server(s) checked
server_dump_info_path | /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden | 127.3.139.65:32961
server_dump_info_path | /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.3.139.66:45373
server_dump_info_path | /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.3.139.67:36601
server_dump_info_path | /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.3.139.68:33975
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.3.139.84:44509 | all 4 server(s) checked
time_source | builtin | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
------------------------+----+---------+---------------+---------+------------+------------------+-------------
TestTable | 1 | HEALTHY | 1 | 1 | 0 | 0 | 0
rebalance_test_table_0 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_1 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_2 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_3 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 6
Third Quartile | 7
Maximum | 7
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 5
Tablets | 13
Replicas | 13
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20240502 07:59:25.661375 3629 log_verifier.cc:126] Checking tablet 05efac4ade434a498c56c44a86f9ccdb
I20240502 07:59:25.668495 3629 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 05efac4ade434a498c56c44a86f9ccdb
I20240502 07:59:25.668537 3629 log_verifier.cc:126] Checking tablet 09cbdcacea3c424493775868ed27362a
I20240502 07:59:25.675139 3629 log_verifier.cc:177] Verified matching terms for 2 ops in tablet 09cbdcacea3c424493775868ed27362a
I20240502 07:59:25.675179 3629 log_verifier.cc:126] Checking tablet 126caa6890ff469c91a4fa20d211ed0d
I20240502 07:59:25.681737 3629 log_verifier.cc:177] Verified matching terms for 2 ops in tablet 126caa6890ff469c91a4fa20d211ed0d
I20240502 07:59:25.681779 3629 log_verifier.cc:126] Checking tablet 276002f558e54354897464ea6e718c20
I20240502 07:59:25.688400 3629 log_verifier.cc:177] Verified matching terms for 2 ops in tablet 276002f558e54354897464ea6e718c20
I20240502 07:59:25.688439 3629 log_verifier.cc:126] Checking tablet 385590337e9d443a825b8a78922cda2a
I20240502 07:59:25.695079 3629 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 385590337e9d443a825b8a78922cda2a
I20240502 07:59:25.695122 3629 log_verifier.cc:126] Checking tablet 57b9f7ff4cd247c9a6d81e3028a1810f
I20240502 07:59:25.701572 3629 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 57b9f7ff4cd247c9a6d81e3028a1810f
I20240502 07:59:25.701613 3629 log_verifier.cc:126] Checking tablet 8327ae04d10c4caaad35984d10c79ffe
I20240502 07:59:25.708263 3629 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 8327ae04d10c4caaad35984d10c79ffe
I20240502 07:59:25.708303 3629 log_verifier.cc:126] Checking tablet bff03e3cb849492f85ee3339da4a5dfa
I20240502 07:59:25.714803 3629 log_verifier.cc:177] Verified matching terms for 1 ops in tablet bff03e3cb849492f85ee3339da4a5dfa
I20240502 07:59:25.714843 3629 log_verifier.cc:126] Checking tablet c90ce897261c4e42a3b0f27bfdad936e
I20240502 07:59:25.721403 3629 log_verifier.cc:177] Verified matching terms for 1 ops in tablet c90ce897261c4e42a3b0f27bfdad936e
I20240502 07:59:25.721442 3629 log_verifier.cc:126] Checking tablet d55bcea3e0e54d3990f494d3ccb3bc68
I20240502 07:59:25.728485 3629 log_verifier.cc:177] Verified matching terms for 2 ops in tablet d55bcea3e0e54d3990f494d3ccb3bc68
I20240502 07:59:25.728552 3629 log_verifier.cc:126] Checking tablet d592eeae00054794becc66f21977cd19
I20240502 07:59:25.736413 3629 log_verifier.cc:177] Verified matching terms for 1 ops in tablet d592eeae00054794becc66f21977cd19
I20240502 07:59:25.736460 3629 log_verifier.cc:126] Checking tablet f4255a612f6247ee83cc6ac2c07c4c18
I20240502 07:59:25.744313 3629 log_verifier.cc:177] Verified matching terms for 1 ops in tablet f4255a612f6247ee83cc6ac2c07c4c18
I20240502 07:59:25.744359 3629 log_verifier.cc:126] Checking tablet f7948b810fe449a894e3d572c0aacf44
I20240502 07:59:25.750991 3629 log_verifier.cc:177] Verified matching terms for 1 ops in tablet f7948b810fe449a894e3d572c0aacf44
I20240502 07:59:25.751415 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 17201
I20240502 07:59:25.757388 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 17332
I20240502 07:59:25.762231 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 17858
I20240502 07:59:25.766451 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 17989
I20240502 07:59:25.770364 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 17110
2024-05-02T07:59:25Z chronyd exiting
[ OK ] RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove/5 (6270 ms)
[----------] 1 test from RebalancerAndSingleReplicaTablets (6270 ms total)
[----------] 2 tests from LocationAwareRebalancingParamTest
[ RUN ] LocationAwareRebalancingParamTest.Rebalance/idx7_rf2_t3_l_1_2_1
I20240502 07:59:25.798017 3629 test_util.cc:268] Using random seed: -78111105
I20240502 07:59:25.798658 3629 ts_itest-base.cc:112] Starting cluster with:
I20240502 07:59:25.798689 3629 ts_itest-base.cc:113] --------------
I20240502 07:59:25.798699 3629 ts_itest-base.cc:114] 4 tablet servers
I20240502 07:59:25.798708 3629 ts_itest-base.cc:115] 2 replicas per TS
I20240502 07:59:25.798717 3629 ts_itest-base.cc:116] --------------
2024-05-02T07:59:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-02T07:59:25Z Disabled control of system clock
I20240502 07:59:25.806195 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.139.126:43361
--webserver_interface=127.3.139.126
--webserver_port=0
--builtin_ntp_servers=127.3.139.84:44343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.139.126:43361
--location_mapping_cmd=/tmp/dist-test-taskcXDMw5/build/debug/bin/testdata/assign-location.py --state_store=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/location-assignment.state --map /A:1 --map /B:2 --map /C:1
--tserver_unresponsive_timeout_ms=3000
--enable_range_replica_placement=false
--raft_prepare_replacement_before_eviction=true
--enable_range_replica_placement=false
--master_client_location_assignment_enabled=false
--allow_unsafe_replication_factor with env {}
W20240502 07:59:25.904590 18224 flags.cc:425] Enabled unsafe flag: --allow_unsafe_replication_factor=true
W20240502 07:59:25.904808 18224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:25.904832 18224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:59:25.904872 18224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:25.907789 18224 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:59:25.907848 18224 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240502 07:59:25.907891 18224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:59:25.907922 18224 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240502 07:59:25.907951 18224 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240502 07:59:25.911439 18224 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:44343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal
--allow_unsafe_replication_factor=true
--enable_range_replica_placement=false
--location_mapping_cmd=/tmp/dist-test-taskcXDMw5/build/debug/bin/testdata/assign-location.py --state_store=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/location-assignment.state --map /A:1 --map /B:2 --map /C:1
--ipki_ca_key_size=768
--master_addresses=127.3.139.126:43361
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.139.126:43361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.3.139.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.18224
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:59:25.912166 18224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:59:25.913151 18224 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:59:25.919579 18230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:25.919931 18229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:25.920322 18232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:25.920697 18224 server_base.cc:1034] running on GCE node
I20240502 07:59:25.921106 18224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:59:25.921922 18224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:59:25.923094 18224 hybrid_clock.cc:648] HybridClock initialized: now 1714636765923069 us; error 34 us; skew 500 ppm
I20240502 07:59:25.924718 18224 webserver.cc:460] Webserver started at http://127.3.139.126:39553/ using document root <none> and password file <none>
I20240502 07:59:25.925204 18224 fs_manager.cc:359] Metadata directory not provided
I20240502 07:59:25.925256 18224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:59:25.925415 18224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:59:25.926858 18224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "42f68f5e34ca4cf6bd0c971248a23f24"
format_stamp: "Formatted at 2024-05-02 07:59:25 on dist-test-slave-twcr"
I20240502 07:59:25.927321 18224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "42f68f5e34ca4cf6bd0c971248a23f24"
format_stamp: "Formatted at 2024-05-02 07:59:25 on dist-test-slave-twcr"
I20240502 07:59:25.930222 18224 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.002s
I20240502 07:59:25.932130 18238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:25.933004 18224 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:25.933102 18224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal
uuid: "42f68f5e34ca4cf6bd0c971248a23f24"
format_stamp: "Formatted at 2024-05-02 07:59:25 on dist-test-slave-twcr"
I20240502 07:59:25.933173 18224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:59:25.949921 18224 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:59:25.950445 18224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:59:25.950577 18224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:59:25.956383 18290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.126:43361 every 8 connection(s)
I20240502 07:59:25.956380 18224 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.126:43361
I20240502 07:59:25.957387 18224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240502 07:59:25.960048 18291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:25.960354 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 18224
I20240502 07:59:25.960510 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240502 07:59:25.964766 18291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42f68f5e34ca4cf6bd0c971248a23f24: Bootstrap starting.
I20240502 07:59:25.966626 18291 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42f68f5e34ca4cf6bd0c971248a23f24: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:25.967376 18291 log.cc:824] T 00000000000000000000000000000000 P 42f68f5e34ca4cf6bd0c971248a23f24: Log is configured to *not* fsync() on all Append() calls
I20240502 07:59:25.968791 18291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42f68f5e34ca4cf6bd0c971248a23f24: No bootstrap required, opened a new log
I20240502 07:59:25.971024 18291 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42f68f5e34ca4cf6bd0c971248a23f24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42f68f5e34ca4cf6bd0c971248a23f24" member_type: VOTER last_known_addr { host: "127.3.139.126" port: 43361 } }
I20240502 07:59:25.971230 18291 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42f68f5e34ca4cf6bd0c971248a23f24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:25.971292 18291 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42f68f5e34ca4cf6bd0c971248a23f24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42f68f5e34ca4cf6bd0c971248a23f24, State: Initialized, Role: FOLLOWER
I20240502 07:59:25.971673 18291 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42f68f5e34ca4cf6bd0c971248a23f24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42f68f5e34ca4cf6bd0c971248a23f24" member_type: VOTER last_known_addr { host: "127.3.139.126" port: 43361 } }
I20240502 07:59:25.971817 18291 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42f68f5e34ca4cf6bd0c971248a23f24 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 07:59:25.971863 18291 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42f68f5e34ca4cf6bd0c971248a23f24 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 07:59:25.971935 18291 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42f68f5e34ca4cf6bd0c971248a23f24 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:25.972493 18291 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42f68f5e34ca4cf6bd0c971248a23f24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42f68f5e34ca4cf6bd0c971248a23f24" member_type: VOTER last_known_addr { host: "127.3.139.126" port: 43361 } }
I20240502 07:59:25.972750 18291 leader_election.cc:302] T 00000000000000000000000000000000 P 42f68f5e34ca4cf6bd0c971248a23f24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42f68f5e34ca4cf6bd0c971248a23f24; no voters:
I20240502 07:59:25.972944 18291 leader_election.cc:288] T 00000000000000000000000000000000 P 42f68f5e34ca4cf6bd0c971248a23f24 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 07:59:25.973031 18296 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42f68f5e34ca4cf6bd0c971248a23f24 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:25.973212 18296 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42f68f5e34ca4cf6bd0c971248a23f24 [term 1 LEADER]: Becoming Leader. State: Replica: 42f68f5e34ca4cf6bd0c971248a23f24, State: Running, Role: LEADER
I20240502 07:59:25.973450 18296 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42f68f5e34ca4cf6bd0c971248a23f24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42f68f5e34ca4cf6bd0c971248a23f24" member_type: VOTER last_known_addr { host: "127.3.139.126" port: 43361 } }
I20240502 07:59:25.973836 18291 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42f68f5e34ca4cf6bd0c971248a23f24 [sys.catalog]: configured and running, proceeding with master startup.
I20240502 07:59:25.974927 18297 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42f68f5e34ca4cf6bd0c971248a23f24 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42f68f5e34ca4cf6bd0c971248a23f24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42f68f5e34ca4cf6bd0c971248a23f24" member_type: VOTER last_known_addr { host: "127.3.139.126" port: 43361 } } }
I20240502 07:59:25.975081 18297 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42f68f5e34ca4cf6bd0c971248a23f24 [sys.catalog]: This master's current role is: LEADER
I20240502 07:59:25.975299 18298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42f68f5e34ca4cf6bd0c971248a23f24 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42f68f5e34ca4cf6bd0c971248a23f24. Latest consensus state: current_term: 1 leader_uuid: "42f68f5e34ca4cf6bd0c971248a23f24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42f68f5e34ca4cf6bd0c971248a23f24" member_type: VOTER last_known_addr { host: "127.3.139.126" port: 43361 } } }
I20240502 07:59:25.975371 18298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42f68f5e34ca4cf6bd0c971248a23f24 [sys.catalog]: This master's current role is: LEADER
I20240502 07:59:25.975929 18303 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240502 07:59:25.978449 18303 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240502 07:59:25.982513 18303 catalog_manager.cc:1347] Generated new cluster ID: 869ef84489874cdea78aa7387fb0fd02
I20240502 07:59:25.982574 18303 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240502 07:59:25.994894 18303 catalog_manager.cc:1370] Generated new certificate authority record
I20240502 07:59:25.995649 18303 catalog_manager.cc:1504] Loading token signing keys...
I20240502 07:59:25.999512 18303 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 42f68f5e34ca4cf6bd0c971248a23f24: Generated new TSK 0
I20240502 07:59:26.000130 18303 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240502 07:59:26.011304 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.65:0
--local_ip_for_outbound_sockets=127.3.139.65
--webserver_interface=127.3.139.65
--webserver_port=0
--tserver_master_addrs=127.3.139.126:43361
--builtin_ntp_servers=127.3.139.84:44343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240502 07:59:26.109129 18315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:26.109306 18315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:59:26.109346 18315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:26.112223 18315 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:59:26.112277 18315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:59:26.112342 18315 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.65
I20240502 07:59:26.115746 18315 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:44343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.3.139.65
--webserver_port=0
--tserver_master_addrs=127.3.139.126:43361
--never_fsync=true
--heap_profile_path=/tmp/kudu.18315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.65
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:59:26.116492 18315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:59:26.117487 18315 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:59:26.125309 18321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:26.125630 18320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:26.125984 18323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:26.126376 18315 server_base.cc:1034] running on GCE node
I20240502 07:59:26.126694 18315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:59:26.127199 18315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:59:26.128340 18315 hybrid_clock.cc:648] HybridClock initialized: now 1714636766128325 us; error 27 us; skew 500 ppm
I20240502 07:59:26.129953 18315 webserver.cc:460] Webserver started at http://127.3.139.65:34633/ using document root <none> and password file <none>
I20240502 07:59:26.130430 18315 fs_manager.cc:359] Metadata directory not provided
I20240502 07:59:26.130486 18315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:59:26.130637 18315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:59:26.132156 18315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "d52c06abbcb44881a4837273c1ec8e91"
format_stamp: "Formatted at 2024-05-02 07:59:26 on dist-test-slave-twcr"
I20240502 07:59:26.132579 18315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "d52c06abbcb44881a4837273c1ec8e91"
format_stamp: "Formatted at 2024-05-02 07:59:26 on dist-test-slave-twcr"
I20240502 07:59:26.135464 18315 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20240502 07:59:26.137331 18329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:26.138191 18315 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:26.138283 18315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "d52c06abbcb44881a4837273c1ec8e91"
format_stamp: "Formatted at 2024-05-02 07:59:26 on dist-test-slave-twcr"
I20240502 07:59:26.138353 18315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:59:26.165335 18315 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:59:26.165827 18315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:59:26.165971 18315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:59:26.166425 18315 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:59:26.167251 18315 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:59:26.167296 18315 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:26.167344 18315 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:59:26.167371 18315 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:26.175349 18315 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.65:40139
I20240502 07:59:26.175477 18442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.65:40139 every 8 connection(s)
I20240502 07:59:26.176347 18315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240502 07:59:26.185029 18443 heartbeater.cc:347] Connected to a master server at 127.3.139.126:43361
I20240502 07:59:26.185225 18443 heartbeater.cc:464] Registering TS with master...
I20240502 07:59:26.185675 18443 heartbeater.cc:511] Master 127.3.139.126:43361 requested a full tablet report, sending...
I20240502 07:59:26.186219 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 18315
I20240502 07:59:26.186342 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240502 07:59:26.190690 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.66:0
--local_ip_for_outbound_sockets=127.3.139.66
--webserver_interface=127.3.139.66
--webserver_port=0
--tserver_master_addrs=127.3.139.126:43361
--builtin_ntp_servers=127.3.139.84:44343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 07:59:26.219465 18255 ts_manager.cc:194] Registered new tserver with Master: d52c06abbcb44881a4837273c1ec8e91 (127.3.139.65:40139)
I20240502 07:59:26.221194 18255 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.65:57775
W20240502 07:59:26.288815 18447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:26.288983 18447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:59:26.289023 18447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:26.291833 18447 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:59:26.291888 18447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:59:26.291952 18447 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.66
I20240502 07:59:26.295255 18447 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:44343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.3.139.66
--webserver_port=0
--tserver_master_addrs=127.3.139.126:43361
--never_fsync=true
--heap_profile_path=/tmp/kudu.18447
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.66
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:59:26.295902 18447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:59:26.296792 18447 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:59:26.303966 18455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:26.304297 18452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:26.304651 18453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:26.305037 18447 server_base.cc:1034] running on GCE node
I20240502 07:59:26.305370 18447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:59:26.305836 18447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:59:26.306985 18447 hybrid_clock.cc:648] HybridClock initialized: now 1714636766306968 us; error 31 us; skew 500 ppm
I20240502 07:59:26.308568 18447 webserver.cc:460] Webserver started at http://127.3.139.66:34827/ using document root <none> and password file <none>
I20240502 07:59:26.309041 18447 fs_manager.cc:359] Metadata directory not provided
I20240502 07:59:26.309094 18447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:59:26.309245 18447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:59:26.310642 18447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "6fd33d34de9c463183177f1a13d4f851"
format_stamp: "Formatted at 2024-05-02 07:59:26 on dist-test-slave-twcr"
I20240502 07:59:26.311085 18447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "6fd33d34de9c463183177f1a13d4f851"
format_stamp: "Formatted at 2024-05-02 07:59:26 on dist-test-slave-twcr"
I20240502 07:59:26.313874 18447 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20240502 07:59:26.315707 18461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:26.316550 18447 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:26.316650 18447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "6fd33d34de9c463183177f1a13d4f851"
format_stamp: "Formatted at 2024-05-02 07:59:26 on dist-test-slave-twcr"
I20240502 07:59:26.316720 18447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:59:26.338071 18447 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:59:26.338505 18447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:59:26.338632 18447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:59:26.339080 18447 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:59:26.339815 18447 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:59:26.339879 18447 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:26.339924 18447 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:59:26.339951 18447 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:26.347376 18447 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.66:35635
I20240502 07:59:26.347424 18574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.66:35635 every 8 connection(s)
I20240502 07:59:26.348140 18447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240502 07:59:26.355528 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 18447
I20240502 07:59:26.355664 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240502 07:59:26.355722 18575 heartbeater.cc:347] Connected to a master server at 127.3.139.126:43361
I20240502 07:59:26.355927 18575 heartbeater.cc:464] Registering TS with master...
I20240502 07:59:26.356420 18575 heartbeater.cc:511] Master 127.3.139.126:43361 requested a full tablet report, sending...
I20240502 07:59:26.357733 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.67:0
--local_ip_for_outbound_sockets=127.3.139.67
--webserver_interface=127.3.139.67
--webserver_port=0
--tserver_master_addrs=127.3.139.126:43361
--builtin_ntp_servers=127.3.139.84:44343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 07:59:26.387696 18255 ts_manager.cc:194] Registered new tserver with Master: 6fd33d34de9c463183177f1a13d4f851 (127.3.139.66:35635)
I20240502 07:59:26.388628 18255 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.66:41611
W20240502 07:59:26.456583 18579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:26.456760 18579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:59:26.456799 18579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:26.459681 18579 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:59:26.459730 18579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:59:26.459791 18579 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.67
I20240502 07:59:26.463150 18579 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:44343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.3.139.67
--webserver_port=0
--tserver_master_addrs=127.3.139.126:43361
--never_fsync=true
--heap_profile_path=/tmp/kudu.18579
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.67
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:59:26.463835 18579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:59:26.464872 18579 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:59:26.472467 18587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:26.472769 18584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:26.473104 18585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:26.473517 18579 server_base.cc:1034] running on GCE node
I20240502 07:59:26.473834 18579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:59:26.474324 18579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:59:26.475533 18579 hybrid_clock.cc:648] HybridClock initialized: now 1714636766475516 us; error 27 us; skew 500 ppm
I20240502 07:59:26.477229 18579 webserver.cc:460] Webserver started at http://127.3.139.67:46239/ using document root <none> and password file <none>
I20240502 07:59:26.477697 18579 fs_manager.cc:359] Metadata directory not provided
I20240502 07:59:26.477746 18579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:59:26.477887 18579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:59:26.479343 18579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "db667da4313e43e2917b5acc7e6f645d"
format_stamp: "Formatted at 2024-05-02 07:59:26 on dist-test-slave-twcr"
I20240502 07:59:26.479743 18579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "db667da4313e43e2917b5acc7e6f645d"
format_stamp: "Formatted at 2024-05-02 07:59:26 on dist-test-slave-twcr"
I20240502 07:59:26.482610 18579 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240502 07:59:26.484555 18593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:26.485518 18579 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:26.485612 18579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "db667da4313e43e2917b5acc7e6f645d"
format_stamp: "Formatted at 2024-05-02 07:59:26 on dist-test-slave-twcr"
I20240502 07:59:26.485693 18579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:59:26.495244 18579 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:59:26.495693 18579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:59:26.495817 18579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:59:26.496253 18579 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:59:26.496994 18579 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:59:26.497037 18579 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:26.497083 18579 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:59:26.497112 18579 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:26.504936 18579 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.67:33695
I20240502 07:59:26.504964 18706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.67:33695 every 8 connection(s)
I20240502 07:59:26.505718 18579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240502 07:59:26.512378 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 18579
I20240502 07:59:26.512501 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240502 07:59:26.513274 18707 heartbeater.cc:347] Connected to a master server at 127.3.139.126:43361
I20240502 07:59:26.513489 18707 heartbeater.cc:464] Registering TS with master...
I20240502 07:59:26.514045 18707 heartbeater.cc:511] Master 127.3.139.126:43361 requested a full tablet report, sending...
I20240502 07:59:26.514504 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.68:0
--local_ip_for_outbound_sockets=127.3.139.68
--webserver_interface=127.3.139.68
--webserver_port=0
--tserver_master_addrs=127.3.139.126:43361
--builtin_ntp_servers=127.3.139.84:44343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 07:59:26.546942 18255 ts_manager.cc:194] Registered new tserver with Master: db667da4313e43e2917b5acc7e6f645d (127.3.139.67:33695)
I20240502 07:59:26.547917 18255 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.67:52123
W20240502 07:59:26.617300 18710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:26.617491 18710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:59:26.617528 18710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:26.620465 18710 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:59:26.620532 18710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:59:26.620595 18710 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.68
I20240502 07:59:26.623994 18710 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:44343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.68:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.3.139.68
--webserver_port=0
--tserver_master_addrs=127.3.139.126:43361
--never_fsync=true
--heap_profile_path=/tmp/kudu.18710
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.68
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:59:26.624722 18710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:59:26.625705 18710 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:59:26.633183 18717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:26.633527 18716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:26.633927 18719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:26.634323 18710 server_base.cc:1034] running on GCE node
I20240502 07:59:26.634626 18710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:59:26.635135 18710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:59:26.636281 18710 hybrid_clock.cc:648] HybridClock initialized: now 1714636766636269 us; error 23 us; skew 500 ppm
I20240502 07:59:26.638037 18710 webserver.cc:460] Webserver started at http://127.3.139.68:41425/ using document root <none> and password file <none>
I20240502 07:59:26.638517 18710 fs_manager.cc:359] Metadata directory not provided
I20240502 07:59:26.638568 18710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:59:26.638731 18710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:59:26.640249 18710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "dbe891b2f04e454db1c2b5de528e01c7"
format_stamp: "Formatted at 2024-05-02 07:59:26 on dist-test-slave-twcr"
I20240502 07:59:26.640668 18710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "dbe891b2f04e454db1c2b5de528e01c7"
format_stamp: "Formatted at 2024-05-02 07:59:26 on dist-test-slave-twcr"
I20240502 07:59:26.643790 18710 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 07:59:26.645748 18725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:26.646661 18710 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:26.646756 18710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "dbe891b2f04e454db1c2b5de528e01c7"
format_stamp: "Formatted at 2024-05-02 07:59:26 on dist-test-slave-twcr"
I20240502 07:59:26.646828 18710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:59:26.658293 18710 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:59:26.658751 18710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:59:26.658892 18710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:59:26.659358 18710 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:59:26.660127 18710 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:59:26.660171 18710 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:26.660215 18710 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:59:26.660241 18710 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:26.668035 18710 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.68:43983
I20240502 07:59:26.668061 18838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.68:43983 every 8 connection(s)
I20240502 07:59:26.668766 18710 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240502 07:59:26.669622 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 18710
I20240502 07:59:26.669752 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240502 07:59:26.676864 18839 heartbeater.cc:347] Connected to a master server at 127.3.139.126:43361
I20240502 07:59:26.677053 18839 heartbeater.cc:464] Registering TS with master...
I20240502 07:59:26.677505 18839 heartbeater.cc:511] Master 127.3.139.126:43361 requested a full tablet report, sending...
I20240502 07:59:26.709241 18255 ts_manager.cc:194] Registered new tserver with Master: dbe891b2f04e454db1c2b5de528e01c7 (127.3.139.68:43983)
I20240502 07:59:26.709872 18255 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.68:48155
I20240502 07:59:26.718180 3629 external_mini_cluster.cc:930] 4 TS(s) registered with all masters
I20240502 07:59:26.726809 3629 external_mini_cluster.cc:930] 4 TS(s) registered with all masters
I20240502 07:59:26.726872 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 18447
I20240502 07:59:26.732019 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 18579
I20240502 07:59:27.223526 18443 heartbeater.cc:503] Master 127.3.139.126:43361 was elected leader, sending a full tablet report...
I20240502 07:59:27.711783 18839 heartbeater.cc:503] Master 127.3.139.126:43361 was elected leader, sending a full tablet report...
I20240502 07:59:30.488214 18255 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51750:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 2
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240502 07:59:30.491328 18255 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_0 in case of a server failure: 3 tablet servers would be needed, 2 are available. Consider bringing up more tablet servers.
I20240502 07:59:30.506938 18772 tablet_service.cc:1467] Processing CreateTablet for tablet 58aa0644416e4e698766581b577d21d0 (DEFAULT_TABLE table=rebalance_test_table_0 [id=d0fba7f1dfcc455d96585f048abd07a2]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:30.506938 18773 tablet_service.cc:1467] Processing CreateTablet for tablet d8c026e315ac4622b77b618de5c0b09f (DEFAULT_TABLE table=rebalance_test_table_0 [id=d0fba7f1dfcc455d96585f048abd07a2]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:30.507380 18376 tablet_service.cc:1467] Processing CreateTablet for tablet 58aa0644416e4e698766581b577d21d0 (DEFAULT_TABLE table=rebalance_test_table_0 [id=d0fba7f1dfcc455d96585f048abd07a2]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:30.507380 18377 tablet_service.cc:1467] Processing CreateTablet for tablet d8c026e315ac4622b77b618de5c0b09f (DEFAULT_TABLE table=rebalance_test_table_0 [id=d0fba7f1dfcc455d96585f048abd07a2]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:30.507488 18375 tablet_service.cc:1467] Processing CreateTablet for tablet 1efba9718b4b466786f6d1a93be119d9 (DEFAULT_TABLE table=rebalance_test_table_0 [id=d0fba7f1dfcc455d96585f048abd07a2]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:30.507944 18773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8c026e315ac4622b77b618de5c0b09f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:30.506940 18771 tablet_service.cc:1467] Processing CreateTablet for tablet 1efba9718b4b466786f6d1a93be119d9 (DEFAULT_TABLE table=rebalance_test_table_0 [id=d0fba7f1dfcc455d96585f048abd07a2]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:30.508294 18771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1efba9718b4b466786f6d1a93be119d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:30.508306 18375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1efba9718b4b466786f6d1a93be119d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:30.508559 18772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58aa0644416e4e698766581b577d21d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:30.509042 18377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8c026e315ac4622b77b618de5c0b09f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:30.509153 18376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58aa0644416e4e698766581b577d21d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:30.513931 18857 tablet_bootstrap.cc:492] T 1efba9718b4b466786f6d1a93be119d9 P d52c06abbcb44881a4837273c1ec8e91: Bootstrap starting.
I20240502 07:59:30.515120 18856 tablet_bootstrap.cc:492] T 58aa0644416e4e698766581b577d21d0 P dbe891b2f04e454db1c2b5de528e01c7: Bootstrap starting.
I20240502 07:59:30.516050 18857 tablet_bootstrap.cc:654] T 1efba9718b4b466786f6d1a93be119d9 P d52c06abbcb44881a4837273c1ec8e91: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:30.516834 18857 log.cc:824] T 1efba9718b4b466786f6d1a93be119d9 P d52c06abbcb44881a4837273c1ec8e91: Log is configured to *not* fsync() on all Append() calls
I20240502 07:59:30.516940 18856 tablet_bootstrap.cc:654] T 58aa0644416e4e698766581b577d21d0 P dbe891b2f04e454db1c2b5de528e01c7: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:30.517953 18856 log.cc:824] T 58aa0644416e4e698766581b577d21d0 P dbe891b2f04e454db1c2b5de528e01c7: Log is configured to *not* fsync() on all Append() calls
I20240502 07:59:30.518451 18857 tablet_bootstrap.cc:492] T 1efba9718b4b466786f6d1a93be119d9 P d52c06abbcb44881a4837273c1ec8e91: No bootstrap required, opened a new log
I20240502 07:59:30.518616 18857 ts_tablet_manager.cc:1367] T 1efba9718b4b466786f6d1a93be119d9 P d52c06abbcb44881a4837273c1ec8e91: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20240502 07:59:30.519500 18856 tablet_bootstrap.cc:492] T 58aa0644416e4e698766581b577d21d0 P dbe891b2f04e454db1c2b5de528e01c7: No bootstrap required, opened a new log
I20240502 07:59:30.519652 18856 ts_tablet_manager.cc:1367] T 58aa0644416e4e698766581b577d21d0 P dbe891b2f04e454db1c2b5de528e01c7: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20240502 07:59:30.521921 18857 raft_consensus.cc:348] T 1efba9718b4b466786f6d1a93be119d9 P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } }
I20240502 07:59:30.522157 18857 raft_consensus.cc:374] T 1efba9718b4b466786f6d1a93be119d9 P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:30.522221 18857 raft_consensus.cc:729] T 1efba9718b4b466786f6d1a93be119d9 P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d52c06abbcb44881a4837273c1ec8e91, State: Initialized, Role: FOLLOWER
I20240502 07:59:30.522500 18856 raft_consensus.cc:348] T 58aa0644416e4e698766581b577d21d0 P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } }
I20240502 07:59:30.522675 18856 raft_consensus.cc:374] T 58aa0644416e4e698766581b577d21d0 P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:30.522631 18857 consensus_queue.cc:260] T 1efba9718b4b466786f6d1a93be119d9 P d52c06abbcb44881a4837273c1ec8e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } }
I20240502 07:59:30.522722 18856 raft_consensus.cc:729] T 58aa0644416e4e698766581b577d21d0 P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbe891b2f04e454db1c2b5de528e01c7, State: Initialized, Role: FOLLOWER
I20240502 07:59:30.523120 18856 consensus_queue.cc:260] T 58aa0644416e4e698766581b577d21d0 P dbe891b2f04e454db1c2b5de528e01c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } }
I20240502 07:59:30.523471 18857 ts_tablet_manager.cc:1398] T 1efba9718b4b466786f6d1a93be119d9 P d52c06abbcb44881a4837273c1ec8e91: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20240502 07:59:30.523756 18857 tablet_bootstrap.cc:492] T 58aa0644416e4e698766581b577d21d0 P d52c06abbcb44881a4837273c1ec8e91: Bootstrap starting.
I20240502 07:59:30.523876 18856 ts_tablet_manager.cc:1398] T 58aa0644416e4e698766581b577d21d0 P dbe891b2f04e454db1c2b5de528e01c7: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20240502 07:59:30.524164 18856 tablet_bootstrap.cc:492] T d8c026e315ac4622b77b618de5c0b09f P dbe891b2f04e454db1c2b5de528e01c7: Bootstrap starting.
I20240502 07:59:30.525032 18856 tablet_bootstrap.cc:654] T d8c026e315ac4622b77b618de5c0b09f P dbe891b2f04e454db1c2b5de528e01c7: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:30.525637 18857 tablet_bootstrap.cc:654] T 58aa0644416e4e698766581b577d21d0 P d52c06abbcb44881a4837273c1ec8e91: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:30.526101 18856 tablet_bootstrap.cc:492] T d8c026e315ac4622b77b618de5c0b09f P dbe891b2f04e454db1c2b5de528e01c7: No bootstrap required, opened a new log
I20240502 07:59:30.526178 18856 ts_tablet_manager.cc:1367] T d8c026e315ac4622b77b618de5c0b09f P dbe891b2f04e454db1c2b5de528e01c7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:30.526686 18856 raft_consensus.cc:348] T d8c026e315ac4622b77b618de5c0b09f P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } }
I20240502 07:59:30.526767 18856 raft_consensus.cc:374] T d8c026e315ac4622b77b618de5c0b09f P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:30.526795 18856 raft_consensus.cc:729] T d8c026e315ac4622b77b618de5c0b09f P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbe891b2f04e454db1c2b5de528e01c7, State: Initialized, Role: FOLLOWER
I20240502 07:59:30.526896 18856 consensus_queue.cc:260] T d8c026e315ac4622b77b618de5c0b09f P dbe891b2f04e454db1c2b5de528e01c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } }
I20240502 07:59:30.526964 18857 tablet_bootstrap.cc:492] T 58aa0644416e4e698766581b577d21d0 P d52c06abbcb44881a4837273c1ec8e91: No bootstrap required, opened a new log
I20240502 07:59:30.527024 18857 ts_tablet_manager.cc:1367] T 58aa0644416e4e698766581b577d21d0 P d52c06abbcb44881a4837273c1ec8e91: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:59:30.527292 18856 ts_tablet_manager.cc:1398] T d8c026e315ac4622b77b618de5c0b09f P dbe891b2f04e454db1c2b5de528e01c7: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:30.527432 18856 tablet_bootstrap.cc:492] T 1efba9718b4b466786f6d1a93be119d9 P dbe891b2f04e454db1c2b5de528e01c7: Bootstrap starting.
I20240502 07:59:30.527534 18857 raft_consensus.cc:348] T 58aa0644416e4e698766581b577d21d0 P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } }
I20240502 07:59:30.527609 18857 raft_consensus.cc:374] T 58aa0644416e4e698766581b577d21d0 P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:30.527637 18857 raft_consensus.cc:729] T 58aa0644416e4e698766581b577d21d0 P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d52c06abbcb44881a4837273c1ec8e91, State: Initialized, Role: FOLLOWER
I20240502 07:59:30.527748 18857 consensus_queue.cc:260] T 58aa0644416e4e698766581b577d21d0 P d52c06abbcb44881a4837273c1ec8e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } }
I20240502 07:59:30.528014 18857 ts_tablet_manager.cc:1398] T 58aa0644416e4e698766581b577d21d0 P d52c06abbcb44881a4837273c1ec8e91: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:30.528172 18857 tablet_bootstrap.cc:492] T d8c026e315ac4622b77b618de5c0b09f P d52c06abbcb44881a4837273c1ec8e91: Bootstrap starting.
I20240502 07:59:30.528188 18856 tablet_bootstrap.cc:654] T 1efba9718b4b466786f6d1a93be119d9 P dbe891b2f04e454db1c2b5de528e01c7: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:30.528923 18857 tablet_bootstrap.cc:654] T d8c026e315ac4622b77b618de5c0b09f P d52c06abbcb44881a4837273c1ec8e91: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:30.529366 18856 tablet_bootstrap.cc:492] T 1efba9718b4b466786f6d1a93be119d9 P dbe891b2f04e454db1c2b5de528e01c7: No bootstrap required, opened a new log
I20240502 07:59:30.529431 18856 ts_tablet_manager.cc:1367] T 1efba9718b4b466786f6d1a93be119d9 P dbe891b2f04e454db1c2b5de528e01c7: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 07:59:30.529775 18856 raft_consensus.cc:348] T 1efba9718b4b466786f6d1a93be119d9 P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } }
I20240502 07:59:30.529893 18857 tablet_bootstrap.cc:492] T d8c026e315ac4622b77b618de5c0b09f P d52c06abbcb44881a4837273c1ec8e91: No bootstrap required, opened a new log
I20240502 07:59:30.529893 18856 raft_consensus.cc:374] T 1efba9718b4b466786f6d1a93be119d9 P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:30.529942 18856 raft_consensus.cc:729] T 1efba9718b4b466786f6d1a93be119d9 P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbe891b2f04e454db1c2b5de528e01c7, State: Initialized, Role: FOLLOWER
I20240502 07:59:30.529979 18857 ts_tablet_manager.cc:1367] T d8c026e315ac4622b77b618de5c0b09f P d52c06abbcb44881a4837273c1ec8e91: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 07:59:30.530087 18856 consensus_queue.cc:260] T 1efba9718b4b466786f6d1a93be119d9 P dbe891b2f04e454db1c2b5de528e01c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } }
I20240502 07:59:30.530301 18856 ts_tablet_manager.cc:1398] T 1efba9718b4b466786f6d1a93be119d9 P dbe891b2f04e454db1c2b5de528e01c7: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:30.530400 18857 raft_consensus.cc:348] T d8c026e315ac4622b77b618de5c0b09f P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } }
I20240502 07:59:30.530496 18857 raft_consensus.cc:374] T d8c026e315ac4622b77b618de5c0b09f P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:30.530534 18857 raft_consensus.cc:729] T d8c026e315ac4622b77b618de5c0b09f P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d52c06abbcb44881a4837273c1ec8e91, State: Initialized, Role: FOLLOWER
I20240502 07:59:30.530642 18857 consensus_queue.cc:260] T d8c026e315ac4622b77b618de5c0b09f P d52c06abbcb44881a4837273c1ec8e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } }
I20240502 07:59:30.530927 18857 ts_tablet_manager.cc:1398] T d8c026e315ac4622b77b618de5c0b09f P d52c06abbcb44881a4837273c1ec8e91: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:30.617396 18860 raft_consensus.cc:482] T 58aa0644416e4e698766581b577d21d0 P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:30.617583 18860 raft_consensus.cc:504] T 58aa0644416e4e698766581b577d21d0 P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } }
I20240502 07:59:30.618531 18860 leader_election.cc:288] T 58aa0644416e4e698766581b577d21d0 P d52c06abbcb44881a4837273c1ec8e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dbe891b2f04e454db1c2b5de528e01c7 (127.3.139.68:43983)
I20240502 07:59:30.622138 18793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58aa0644416e4e698766581b577d21d0" candidate_uuid: "d52c06abbcb44881a4837273c1ec8e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbe891b2f04e454db1c2b5de528e01c7" is_pre_election: true
I20240502 07:59:30.622426 18793 raft_consensus.cc:2454] T 58aa0644416e4e698766581b577d21d0 P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d52c06abbcb44881a4837273c1ec8e91 in term 0.
I20240502 07:59:30.622866 18332 leader_election.cc:302] T 58aa0644416e4e698766581b577d21d0 P d52c06abbcb44881a4837273c1ec8e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: d52c06abbcb44881a4837273c1ec8e91, dbe891b2f04e454db1c2b5de528e01c7; no voters:
I20240502 07:59:30.623235 18860 raft_consensus.cc:2781] T 58aa0644416e4e698766581b577d21d0 P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:30.623329 18860 raft_consensus.cc:482] T 58aa0644416e4e698766581b577d21d0 P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:30.623379 18860 raft_consensus.cc:3037] T 58aa0644416e4e698766581b577d21d0 P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:30.624193 18860 raft_consensus.cc:504] T 58aa0644416e4e698766581b577d21d0 P d52c06abbcb44881a4837273c1ec8e91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } }
I20240502 07:59:30.624434 18860 leader_election.cc:288] T 58aa0644416e4e698766581b577d21d0 P d52c06abbcb44881a4837273c1ec8e91 [CANDIDATE]: Term 1 election: Requested vote from peers dbe891b2f04e454db1c2b5de528e01c7 (127.3.139.68:43983)
I20240502 07:59:30.624783 18793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58aa0644416e4e698766581b577d21d0" candidate_uuid: "d52c06abbcb44881a4837273c1ec8e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbe891b2f04e454db1c2b5de528e01c7"
I20240502 07:59:30.624893 18793 raft_consensus.cc:3037] T 58aa0644416e4e698766581b577d21d0 P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:30.625959 18793 raft_consensus.cc:2454] T 58aa0644416e4e698766581b577d21d0 P dbe891b2f04e454db1c2b5de528e01c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d52c06abbcb44881a4837273c1ec8e91 in term 1.
I20240502 07:59:30.626219 18332 leader_election.cc:302] T 58aa0644416e4e698766581b577d21d0 P d52c06abbcb44881a4837273c1ec8e91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: d52c06abbcb44881a4837273c1ec8e91, dbe891b2f04e454db1c2b5de528e01c7; no voters:
I20240502 07:59:30.626366 18860 raft_consensus.cc:2781] T 58aa0644416e4e698766581b577d21d0 P d52c06abbcb44881a4837273c1ec8e91 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:30.626539 18860 raft_consensus.cc:686] T 58aa0644416e4e698766581b577d21d0 P d52c06abbcb44881a4837273c1ec8e91 [term 1 LEADER]: Becoming Leader. State: Replica: d52c06abbcb44881a4837273c1ec8e91, State: Running, Role: LEADER
I20240502 07:59:30.626823 18860 consensus_queue.cc:237] T 58aa0644416e4e698766581b577d21d0 P d52c06abbcb44881a4837273c1ec8e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } }
I20240502 07:59:30.629429 18254 catalog_manager.cc:5484] T 58aa0644416e4e698766581b577d21d0 P d52c06abbcb44881a4837273c1ec8e91 reported cstate change: term changed from 0 to 1, leader changed from <none> to d52c06abbcb44881a4837273c1ec8e91 (127.3.139.65). New cstate: current_term: 1 leader_uuid: "d52c06abbcb44881a4837273c1ec8e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } health_report { overall_health: UNKNOWN } } }
W20240502 07:59:30.671937 18840 tablet.cc:2369] T 1efba9718b4b466786f6d1a93be119d9 P dbe891b2f04e454db1c2b5de528e01c7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240502 07:59:30.679925 18444 tablet.cc:2369] T 1efba9718b4b466786f6d1a93be119d9 P d52c06abbcb44881a4837273c1ec8e91: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 07:59:30.858127 18861 raft_consensus.cc:482] T d8c026e315ac4622b77b618de5c0b09f P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:30.858255 18861 raft_consensus.cc:504] T d8c026e315ac4622b77b618de5c0b09f P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } }
I20240502 07:59:30.859206 18861 leader_election.cc:288] T d8c026e315ac4622b77b618de5c0b09f P dbe891b2f04e454db1c2b5de528e01c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d52c06abbcb44881a4837273c1ec8e91 (127.3.139.65:40139)
I20240502 07:59:30.862862 18397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8c026e315ac4622b77b618de5c0b09f" candidate_uuid: "dbe891b2f04e454db1c2b5de528e01c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d52c06abbcb44881a4837273c1ec8e91" is_pre_election: true
I20240502 07:59:30.863080 18397 raft_consensus.cc:2454] T d8c026e315ac4622b77b618de5c0b09f P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe891b2f04e454db1c2b5de528e01c7 in term 0.
I20240502 07:59:30.863456 18729 leader_election.cc:302] T d8c026e315ac4622b77b618de5c0b09f P dbe891b2f04e454db1c2b5de528e01c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: d52c06abbcb44881a4837273c1ec8e91, dbe891b2f04e454db1c2b5de528e01c7; no voters:
I20240502 07:59:30.863672 18861 raft_consensus.cc:2781] T d8c026e315ac4622b77b618de5c0b09f P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:30.863736 18861 raft_consensus.cc:482] T d8c026e315ac4622b77b618de5c0b09f P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:30.863778 18861 raft_consensus.cc:3037] T d8c026e315ac4622b77b618de5c0b09f P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:30.864548 18861 raft_consensus.cc:504] T d8c026e315ac4622b77b618de5c0b09f P dbe891b2f04e454db1c2b5de528e01c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } }
I20240502 07:59:30.864782 18861 leader_election.cc:288] T d8c026e315ac4622b77b618de5c0b09f P dbe891b2f04e454db1c2b5de528e01c7 [CANDIDATE]: Term 1 election: Requested vote from peers d52c06abbcb44881a4837273c1ec8e91 (127.3.139.65:40139)
I20240502 07:59:30.865118 18397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8c026e315ac4622b77b618de5c0b09f" candidate_uuid: "dbe891b2f04e454db1c2b5de528e01c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d52c06abbcb44881a4837273c1ec8e91"
I20240502 07:59:30.865216 18397 raft_consensus.cc:3037] T d8c026e315ac4622b77b618de5c0b09f P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:30.865931 18397 raft_consensus.cc:2454] T d8c026e315ac4622b77b618de5c0b09f P d52c06abbcb44881a4837273c1ec8e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe891b2f04e454db1c2b5de528e01c7 in term 1.
I20240502 07:59:30.866179 18729 leader_election.cc:302] T d8c026e315ac4622b77b618de5c0b09f P dbe891b2f04e454db1c2b5de528e01c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: d52c06abbcb44881a4837273c1ec8e91, dbe891b2f04e454db1c2b5de528e01c7; no voters:
I20240502 07:59:30.866331 18861 raft_consensus.cc:2781] T d8c026e315ac4622b77b618de5c0b09f P dbe891b2f04e454db1c2b5de528e01c7 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:30.866516 18861 raft_consensus.cc:686] T d8c026e315ac4622b77b618de5c0b09f P dbe891b2f04e454db1c2b5de528e01c7 [term 1 LEADER]: Becoming Leader. State: Replica: dbe891b2f04e454db1c2b5de528e01c7, State: Running, Role: LEADER
I20240502 07:59:30.866827 18861 consensus_queue.cc:237] T d8c026e315ac4622b77b618de5c0b09f P dbe891b2f04e454db1c2b5de528e01c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } }
I20240502 07:59:30.869429 18254 catalog_manager.cc:5484] T d8c026e315ac4622b77b618de5c0b09f P dbe891b2f04e454db1c2b5de528e01c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbe891b2f04e454db1c2b5de528e01c7 (127.3.139.68). New cstate: current_term: 1 leader_uuid: "dbe891b2f04e454db1c2b5de528e01c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:30.890550 18861 raft_consensus.cc:482] T 1efba9718b4b466786f6d1a93be119d9 P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:30.890635 18861 raft_consensus.cc:504] T 1efba9718b4b466786f6d1a93be119d9 P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } }
I20240502 07:59:30.890923 18861 leader_election.cc:288] T 1efba9718b4b466786f6d1a93be119d9 P dbe891b2f04e454db1c2b5de528e01c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d52c06abbcb44881a4837273c1ec8e91 (127.3.139.65:40139)
I20240502 07:59:30.891291 18397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1efba9718b4b466786f6d1a93be119d9" candidate_uuid: "dbe891b2f04e454db1c2b5de528e01c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d52c06abbcb44881a4837273c1ec8e91" is_pre_election: true
I20240502 07:59:30.891397 18397 raft_consensus.cc:2454] T 1efba9718b4b466786f6d1a93be119d9 P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe891b2f04e454db1c2b5de528e01c7 in term 0.
I20240502 07:59:30.891624 18729 leader_election.cc:302] T 1efba9718b4b466786f6d1a93be119d9 P dbe891b2f04e454db1c2b5de528e01c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: d52c06abbcb44881a4837273c1ec8e91, dbe891b2f04e454db1c2b5de528e01c7; no voters:
I20240502 07:59:30.891786 18861 raft_consensus.cc:2781] T 1efba9718b4b466786f6d1a93be119d9 P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:30.891844 18861 raft_consensus.cc:482] T 1efba9718b4b466786f6d1a93be119d9 P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:30.891899 18861 raft_consensus.cc:3037] T 1efba9718b4b466786f6d1a93be119d9 P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:30.892594 18861 raft_consensus.cc:504] T 1efba9718b4b466786f6d1a93be119d9 P dbe891b2f04e454db1c2b5de528e01c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } }
I20240502 07:59:30.892829 18861 leader_election.cc:288] T 1efba9718b4b466786f6d1a93be119d9 P dbe891b2f04e454db1c2b5de528e01c7 [CANDIDATE]: Term 1 election: Requested vote from peers d52c06abbcb44881a4837273c1ec8e91 (127.3.139.65:40139)
I20240502 07:59:30.893110 18397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1efba9718b4b466786f6d1a93be119d9" candidate_uuid: "dbe891b2f04e454db1c2b5de528e01c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d52c06abbcb44881a4837273c1ec8e91"
I20240502 07:59:30.893198 18397 raft_consensus.cc:3037] T 1efba9718b4b466786f6d1a93be119d9 P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:30.893872 18397 raft_consensus.cc:2454] T 1efba9718b4b466786f6d1a93be119d9 P d52c06abbcb44881a4837273c1ec8e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe891b2f04e454db1c2b5de528e01c7 in term 1.
I20240502 07:59:30.894102 18729 leader_election.cc:302] T 1efba9718b4b466786f6d1a93be119d9 P dbe891b2f04e454db1c2b5de528e01c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: d52c06abbcb44881a4837273c1ec8e91, dbe891b2f04e454db1c2b5de528e01c7; no voters:
I20240502 07:59:30.894237 18861 raft_consensus.cc:2781] T 1efba9718b4b466786f6d1a93be119d9 P dbe891b2f04e454db1c2b5de528e01c7 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:30.894320 18861 raft_consensus.cc:686] T 1efba9718b4b466786f6d1a93be119d9 P dbe891b2f04e454db1c2b5de528e01c7 [term 1 LEADER]: Becoming Leader. State: Replica: dbe891b2f04e454db1c2b5de528e01c7, State: Running, Role: LEADER
I20240502 07:59:30.894435 18861 consensus_queue.cc:237] T 1efba9718b4b466786f6d1a93be119d9 P dbe891b2f04e454db1c2b5de528e01c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } }
I20240502 07:59:30.895696 18254 catalog_manager.cc:5484] T 1efba9718b4b466786f6d1a93be119d9 P dbe891b2f04e454db1c2b5de528e01c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbe891b2f04e454db1c2b5de528e01c7 (127.3.139.68). New cstate: current_term: 1 leader_uuid: "dbe891b2f04e454db1c2b5de528e01c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } health_report { overall_health: UNKNOWN } } }
W20240502 07:59:31.041972 18873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:31.042171 18873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:31.045074 18873 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_0'
I20240502 07:59:31.085741 18860 consensus_queue.cc:1035] T 58aa0644416e4e698766581b577d21d0 P d52c06abbcb44881a4837273c1ec8e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:31.130913 18397 raft_consensus.cc:1261] T 1efba9718b4b466786f6d1a93be119d9 P d52c06abbcb44881a4837273c1ec8e91 [term 1 FOLLOWER]: Refusing update from remote peer dbe891b2f04e454db1c2b5de528e01c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:31.131625 18861 consensus_queue.cc:1035] T 1efba9718b4b466786f6d1a93be119d9 P dbe891b2f04e454db1c2b5de528e01c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:31.137631 18396 raft_consensus.cc:1261] T d8c026e315ac4622b77b618de5c0b09f P d52c06abbcb44881a4837273c1ec8e91 [term 1 FOLLOWER]: Refusing update from remote peer dbe891b2f04e454db1c2b5de528e01c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:31.138764 18861 consensus_queue.cc:1035] T d8c026e315ac4622b77b618de5c0b09f P dbe891b2f04e454db1c2b5de528e01c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:31.146519 18902 mvcc.cc:205] Tried to move back new op lower bound from 7023152214562738176 to 7023152213472600064. Current Snapshot: MvccSnapshot[applied={T|T < 7023152214562738176}]
I20240502 07:59:31.146595 18906 mvcc.cc:205] Tried to move back new op lower bound from 7023152214562738176 to 7023152213472600064. Current Snapshot: MvccSnapshot[applied={T|T < 7023152214562738176}]
INSERT report
rows total: 2000
time total: 91.4255 ms
time per row: 0.0457127 ms
I20240502 07:59:31.176434 18249 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51750:
name: "rebalance_test_table_1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 2
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240502 07:59:31.176923 18249 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_1 in case of a server failure: 3 tablet servers would be needed, 2 are available. Consider bringing up more tablet servers.
I20240502 07:59:31.181430 18771 tablet_service.cc:1467] Processing CreateTablet for tablet a53dba987d8d419c9b3c5ed35b02198f (DEFAULT_TABLE table=rebalance_test_table_1 [id=30b6f8cef2c1488fadb7fd8c388373c5]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:31.181691 18771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a53dba987d8d419c9b3c5ed35b02198f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:31.183210 18377 tablet_service.cc:1467] Processing CreateTablet for tablet a53dba987d8d419c9b3c5ed35b02198f (DEFAULT_TABLE table=rebalance_test_table_1 [id=30b6f8cef2c1488fadb7fd8c388373c5]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:31.183543 18377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a53dba987d8d419c9b3c5ed35b02198f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:31.185425 18916 tablet_bootstrap.cc:492] T a53dba987d8d419c9b3c5ed35b02198f P d52c06abbcb44881a4837273c1ec8e91: Bootstrap starting.
I20240502 07:59:31.185735 18772 tablet_service.cc:1467] Processing CreateTablet for tablet bd37e09a008c4e85804e041122ea1c57 (DEFAULT_TABLE table=rebalance_test_table_1 [id=30b6f8cef2c1488fadb7fd8c388373c5]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:31.185962 18772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd37e09a008c4e85804e041122ea1c57. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:31.186079 18375 tablet_service.cc:1467] Processing CreateTablet for tablet d1ee0467bf17461c9692fa887cf79cc2 (DEFAULT_TABLE table=rebalance_test_table_1 [id=30b6f8cef2c1488fadb7fd8c388373c5]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:31.186293 18375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1ee0467bf17461c9692fa887cf79cc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:31.186568 18916 tablet_bootstrap.cc:654] T a53dba987d8d419c9b3c5ed35b02198f P d52c06abbcb44881a4837273c1ec8e91: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:31.187903 18918 tablet_bootstrap.cc:492] T bd37e09a008c4e85804e041122ea1c57 P dbe891b2f04e454db1c2b5de528e01c7: Bootstrap starting.
I20240502 07:59:31.187983 18773 tablet_service.cc:1467] Processing CreateTablet for tablet d1ee0467bf17461c9692fa887cf79cc2 (DEFAULT_TABLE table=rebalance_test_table_1 [id=30b6f8cef2c1488fadb7fd8c388373c5]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:31.188164 18773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1ee0467bf17461c9692fa887cf79cc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:31.188658 18377 tablet_service.cc:1467] Processing CreateTablet for tablet bd37e09a008c4e85804e041122ea1c57 (DEFAULT_TABLE table=rebalance_test_table_1 [id=30b6f8cef2c1488fadb7fd8c388373c5]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:31.188868 18377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd37e09a008c4e85804e041122ea1c57. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:31.189080 18918 tablet_bootstrap.cc:654] T bd37e09a008c4e85804e041122ea1c57 P dbe891b2f04e454db1c2b5de528e01c7: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:31.190971 18916 tablet_bootstrap.cc:492] T a53dba987d8d419c9b3c5ed35b02198f P d52c06abbcb44881a4837273c1ec8e91: No bootstrap required, opened a new log
I20240502 07:59:31.191148 18916 ts_tablet_manager.cc:1367] T a53dba987d8d419c9b3c5ed35b02198f P d52c06abbcb44881a4837273c1ec8e91: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 07:59:31.191624 18916 raft_consensus.cc:348] T a53dba987d8d419c9b3c5ed35b02198f P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } }
I20240502 07:59:31.191680 18918 tablet_bootstrap.cc:492] T bd37e09a008c4e85804e041122ea1c57 P dbe891b2f04e454db1c2b5de528e01c7: No bootstrap required, opened a new log
I20240502 07:59:31.191745 18916 raft_consensus.cc:374] T a53dba987d8d419c9b3c5ed35b02198f P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:31.191753 18918 ts_tablet_manager.cc:1367] T bd37e09a008c4e85804e041122ea1c57 P dbe891b2f04e454db1c2b5de528e01c7: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 07:59:31.191795 18916 raft_consensus.cc:729] T a53dba987d8d419c9b3c5ed35b02198f P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d52c06abbcb44881a4837273c1ec8e91, State: Initialized, Role: FOLLOWER
I20240502 07:59:31.191967 18916 consensus_queue.cc:260] T a53dba987d8d419c9b3c5ed35b02198f P d52c06abbcb44881a4837273c1ec8e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } }
I20240502 07:59:31.192173 18918 raft_consensus.cc:348] T bd37e09a008c4e85804e041122ea1c57 P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } }
I20240502 07:59:31.192282 18918 raft_consensus.cc:374] T bd37e09a008c4e85804e041122ea1c57 P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:31.192328 18916 ts_tablet_manager.cc:1398] T a53dba987d8d419c9b3c5ed35b02198f P d52c06abbcb44881a4837273c1ec8e91: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:31.192332 18918 raft_consensus.cc:729] T bd37e09a008c4e85804e041122ea1c57 P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbe891b2f04e454db1c2b5de528e01c7, State: Initialized, Role: FOLLOWER
I20240502 07:59:31.192453 18918 consensus_queue.cc:260] T bd37e09a008c4e85804e041122ea1c57 P dbe891b2f04e454db1c2b5de528e01c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } }
I20240502 07:59:31.192698 18918 ts_tablet_manager.cc:1398] T bd37e09a008c4e85804e041122ea1c57 P dbe891b2f04e454db1c2b5de528e01c7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:31.192844 18916 tablet_bootstrap.cc:492] T d1ee0467bf17461c9692fa887cf79cc2 P d52c06abbcb44881a4837273c1ec8e91: Bootstrap starting.
I20240502 07:59:31.193734 18916 tablet_bootstrap.cc:654] T d1ee0467bf17461c9692fa887cf79cc2 P d52c06abbcb44881a4837273c1ec8e91: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:31.195068 18918 tablet_bootstrap.cc:492] T d1ee0467bf17461c9692fa887cf79cc2 P dbe891b2f04e454db1c2b5de528e01c7: Bootstrap starting.
I20240502 07:59:31.196100 18918 tablet_bootstrap.cc:654] T d1ee0467bf17461c9692fa887cf79cc2 P dbe891b2f04e454db1c2b5de528e01c7: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:31.197079 18916 tablet_bootstrap.cc:492] T d1ee0467bf17461c9692fa887cf79cc2 P d52c06abbcb44881a4837273c1ec8e91: No bootstrap required, opened a new log
I20240502 07:59:31.197080 18918 tablet_bootstrap.cc:492] T d1ee0467bf17461c9692fa887cf79cc2 P dbe891b2f04e454db1c2b5de528e01c7: No bootstrap required, opened a new log
I20240502 07:59:31.197161 18916 ts_tablet_manager.cc:1367] T d1ee0467bf17461c9692fa887cf79cc2 P d52c06abbcb44881a4837273c1ec8e91: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:59:31.197161 18918 ts_tablet_manager.cc:1367] T d1ee0467bf17461c9692fa887cf79cc2 P dbe891b2f04e454db1c2b5de528e01c7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:31.197554 18916 raft_consensus.cc:348] T d1ee0467bf17461c9692fa887cf79cc2 P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } }
I20240502 07:59:31.197554 18918 raft_consensus.cc:348] T d1ee0467bf17461c9692fa887cf79cc2 P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } }
I20240502 07:59:31.197664 18916 raft_consensus.cc:374] T d1ee0467bf17461c9692fa887cf79cc2 P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:31.197664 18918 raft_consensus.cc:374] T d1ee0467bf17461c9692fa887cf79cc2 P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:31.197705 18916 raft_consensus.cc:729] T d1ee0467bf17461c9692fa887cf79cc2 P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d52c06abbcb44881a4837273c1ec8e91, State: Initialized, Role: FOLLOWER
I20240502 07:59:31.197705 18918 raft_consensus.cc:729] T d1ee0467bf17461c9692fa887cf79cc2 P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbe891b2f04e454db1c2b5de528e01c7, State: Initialized, Role: FOLLOWER
I20240502 07:59:31.197825 18918 consensus_queue.cc:260] T d1ee0467bf17461c9692fa887cf79cc2 P dbe891b2f04e454db1c2b5de528e01c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } }
I20240502 07:59:31.197837 18916 consensus_queue.cc:260] T d1ee0467bf17461c9692fa887cf79cc2 P d52c06abbcb44881a4837273c1ec8e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } }
I20240502 07:59:31.198068 18918 ts_tablet_manager.cc:1398] T d1ee0467bf17461c9692fa887cf79cc2 P dbe891b2f04e454db1c2b5de528e01c7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:31.198288 18918 tablet_bootstrap.cc:492] T a53dba987d8d419c9b3c5ed35b02198f P dbe891b2f04e454db1c2b5de528e01c7: Bootstrap starting.
I20240502 07:59:31.198581 18916 ts_tablet_manager.cc:1398] T d1ee0467bf17461c9692fa887cf79cc2 P d52c06abbcb44881a4837273c1ec8e91: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:31.198733 18916 tablet_bootstrap.cc:492] T bd37e09a008c4e85804e041122ea1c57 P d52c06abbcb44881a4837273c1ec8e91: Bootstrap starting.
I20240502 07:59:31.199247 18918 tablet_bootstrap.cc:654] T a53dba987d8d419c9b3c5ed35b02198f P dbe891b2f04e454db1c2b5de528e01c7: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:31.200183 18916 tablet_bootstrap.cc:654] T bd37e09a008c4e85804e041122ea1c57 P d52c06abbcb44881a4837273c1ec8e91: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:31.200223 18918 tablet_bootstrap.cc:492] T a53dba987d8d419c9b3c5ed35b02198f P dbe891b2f04e454db1c2b5de528e01c7: No bootstrap required, opened a new log
I20240502 07:59:31.200349 18918 ts_tablet_manager.cc:1367] T a53dba987d8d419c9b3c5ed35b02198f P dbe891b2f04e454db1c2b5de528e01c7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:31.200784 18918 raft_consensus.cc:348] T a53dba987d8d419c9b3c5ed35b02198f P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } }
I20240502 07:59:31.200901 18918 raft_consensus.cc:374] T a53dba987d8d419c9b3c5ed35b02198f P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:31.200954 18918 raft_consensus.cc:729] T a53dba987d8d419c9b3c5ed35b02198f P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbe891b2f04e454db1c2b5de528e01c7, State: Initialized, Role: FOLLOWER
I20240502 07:59:31.201071 18918 consensus_queue.cc:260] T a53dba987d8d419c9b3c5ed35b02198f P dbe891b2f04e454db1c2b5de528e01c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } }
I20240502 07:59:31.201218 18916 tablet_bootstrap.cc:492] T bd37e09a008c4e85804e041122ea1c57 P d52c06abbcb44881a4837273c1ec8e91: No bootstrap required, opened a new log
I20240502 07:59:31.201308 18916 ts_tablet_manager.cc:1367] T bd37e09a008c4e85804e041122ea1c57 P d52c06abbcb44881a4837273c1ec8e91: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:31.201340 18918 ts_tablet_manager.cc:1398] T a53dba987d8d419c9b3c5ed35b02198f P dbe891b2f04e454db1c2b5de528e01c7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:31.201676 18916 raft_consensus.cc:348] T bd37e09a008c4e85804e041122ea1c57 P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } }
I20240502 07:59:31.201771 18916 raft_consensus.cc:374] T bd37e09a008c4e85804e041122ea1c57 P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:31.201813 18916 raft_consensus.cc:729] T bd37e09a008c4e85804e041122ea1c57 P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d52c06abbcb44881a4837273c1ec8e91, State: Initialized, Role: FOLLOWER
I20240502 07:59:31.201921 18916 consensus_queue.cc:260] T bd37e09a008c4e85804e041122ea1c57 P d52c06abbcb44881a4837273c1ec8e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } }
I20240502 07:59:31.202236 18916 ts_tablet_manager.cc:1398] T bd37e09a008c4e85804e041122ea1c57 P d52c06abbcb44881a4837273c1ec8e91: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:31.254324 18860 raft_consensus.cc:482] T bd37e09a008c4e85804e041122ea1c57 P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:31.254426 18860 raft_consensus.cc:504] T bd37e09a008c4e85804e041122ea1c57 P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } }
I20240502 07:59:31.254696 18860 leader_election.cc:288] T bd37e09a008c4e85804e041122ea1c57 P d52c06abbcb44881a4837273c1ec8e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dbe891b2f04e454db1c2b5de528e01c7 (127.3.139.68:43983)
I20240502 07:59:31.255038 18793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd37e09a008c4e85804e041122ea1c57" candidate_uuid: "d52c06abbcb44881a4837273c1ec8e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbe891b2f04e454db1c2b5de528e01c7" is_pre_election: true
I20240502 07:59:31.255182 18793 raft_consensus.cc:2454] T bd37e09a008c4e85804e041122ea1c57 P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d52c06abbcb44881a4837273c1ec8e91 in term 0.
I20240502 07:59:31.255432 18332 leader_election.cc:302] T bd37e09a008c4e85804e041122ea1c57 P d52c06abbcb44881a4837273c1ec8e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: d52c06abbcb44881a4837273c1ec8e91, dbe891b2f04e454db1c2b5de528e01c7; no voters:
I20240502 07:59:31.255590 18860 raft_consensus.cc:2781] T bd37e09a008c4e85804e041122ea1c57 P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:31.255641 18860 raft_consensus.cc:482] T bd37e09a008c4e85804e041122ea1c57 P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:31.255682 18860 raft_consensus.cc:3037] T bd37e09a008c4e85804e041122ea1c57 P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:31.256415 18860 raft_consensus.cc:504] T bd37e09a008c4e85804e041122ea1c57 P d52c06abbcb44881a4837273c1ec8e91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } }
I20240502 07:59:31.256644 18860 leader_election.cc:288] T bd37e09a008c4e85804e041122ea1c57 P d52c06abbcb44881a4837273c1ec8e91 [CANDIDATE]: Term 1 election: Requested vote from peers dbe891b2f04e454db1c2b5de528e01c7 (127.3.139.68:43983)
I20240502 07:59:31.256913 18793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd37e09a008c4e85804e041122ea1c57" candidate_uuid: "d52c06abbcb44881a4837273c1ec8e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbe891b2f04e454db1c2b5de528e01c7"
I20240502 07:59:31.257004 18793 raft_consensus.cc:3037] T bd37e09a008c4e85804e041122ea1c57 P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:31.257660 18793 raft_consensus.cc:2454] T bd37e09a008c4e85804e041122ea1c57 P dbe891b2f04e454db1c2b5de528e01c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d52c06abbcb44881a4837273c1ec8e91 in term 1.
I20240502 07:59:31.257905 18332 leader_election.cc:302] T bd37e09a008c4e85804e041122ea1c57 P d52c06abbcb44881a4837273c1ec8e91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: d52c06abbcb44881a4837273c1ec8e91, dbe891b2f04e454db1c2b5de528e01c7; no voters:
I20240502 07:59:31.258046 18860 raft_consensus.cc:2781] T bd37e09a008c4e85804e041122ea1c57 P d52c06abbcb44881a4837273c1ec8e91 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:31.258116 18860 raft_consensus.cc:686] T bd37e09a008c4e85804e041122ea1c57 P d52c06abbcb44881a4837273c1ec8e91 [term 1 LEADER]: Becoming Leader. State: Replica: d52c06abbcb44881a4837273c1ec8e91, State: Running, Role: LEADER
I20240502 07:59:31.258237 18860 consensus_queue.cc:237] T bd37e09a008c4e85804e041122ea1c57 P d52c06abbcb44881a4837273c1ec8e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } }
I20240502 07:59:31.259479 18255 catalog_manager.cc:5484] T bd37e09a008c4e85804e041122ea1c57 P d52c06abbcb44881a4837273c1ec8e91 reported cstate change: term changed from 0 to 1, leader changed from <none> to d52c06abbcb44881a4837273c1ec8e91 (127.3.139.65). New cstate: current_term: 1 leader_uuid: "d52c06abbcb44881a4837273c1ec8e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:31.273288 18860 raft_consensus.cc:482] T a53dba987d8d419c9b3c5ed35b02198f P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:31.273363 18860 raft_consensus.cc:504] T a53dba987d8d419c9b3c5ed35b02198f P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } }
I20240502 07:59:31.273557 18860 leader_election.cc:288] T a53dba987d8d419c9b3c5ed35b02198f P d52c06abbcb44881a4837273c1ec8e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dbe891b2f04e454db1c2b5de528e01c7 (127.3.139.68:43983)
I20240502 07:59:31.273855 18793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a53dba987d8d419c9b3c5ed35b02198f" candidate_uuid: "d52c06abbcb44881a4837273c1ec8e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbe891b2f04e454db1c2b5de528e01c7" is_pre_election: true
I20240502 07:59:31.273945 18793 raft_consensus.cc:2454] T a53dba987d8d419c9b3c5ed35b02198f P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d52c06abbcb44881a4837273c1ec8e91 in term 0.
I20240502 07:59:31.274145 18332 leader_election.cc:302] T a53dba987d8d419c9b3c5ed35b02198f P d52c06abbcb44881a4837273c1ec8e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: d52c06abbcb44881a4837273c1ec8e91, dbe891b2f04e454db1c2b5de528e01c7; no voters:
I20240502 07:59:31.274286 18860 raft_consensus.cc:2781] T a53dba987d8d419c9b3c5ed35b02198f P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:31.274345 18860 raft_consensus.cc:482] T a53dba987d8d419c9b3c5ed35b02198f P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:31.274384 18860 raft_consensus.cc:3037] T a53dba987d8d419c9b3c5ed35b02198f P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:31.274982 18860 raft_consensus.cc:504] T a53dba987d8d419c9b3c5ed35b02198f P d52c06abbcb44881a4837273c1ec8e91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } }
I20240502 07:59:31.275223 18860 leader_election.cc:288] T a53dba987d8d419c9b3c5ed35b02198f P d52c06abbcb44881a4837273c1ec8e91 [CANDIDATE]: Term 1 election: Requested vote from peers dbe891b2f04e454db1c2b5de528e01c7 (127.3.139.68:43983)
I20240502 07:59:31.275547 18793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a53dba987d8d419c9b3c5ed35b02198f" candidate_uuid: "d52c06abbcb44881a4837273c1ec8e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbe891b2f04e454db1c2b5de528e01c7"
I20240502 07:59:31.275629 18793 raft_consensus.cc:3037] T a53dba987d8d419c9b3c5ed35b02198f P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:31.276180 18793 raft_consensus.cc:2454] T a53dba987d8d419c9b3c5ed35b02198f P dbe891b2f04e454db1c2b5de528e01c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d52c06abbcb44881a4837273c1ec8e91 in term 1.
I20240502 07:59:31.276422 18332 leader_election.cc:302] T a53dba987d8d419c9b3c5ed35b02198f P d52c06abbcb44881a4837273c1ec8e91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: d52c06abbcb44881a4837273c1ec8e91, dbe891b2f04e454db1c2b5de528e01c7; no voters:
I20240502 07:59:31.276575 18860 raft_consensus.cc:2781] T a53dba987d8d419c9b3c5ed35b02198f P d52c06abbcb44881a4837273c1ec8e91 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:31.276659 18860 raft_consensus.cc:686] T a53dba987d8d419c9b3c5ed35b02198f P d52c06abbcb44881a4837273c1ec8e91 [term 1 LEADER]: Becoming Leader. State: Replica: d52c06abbcb44881a4837273c1ec8e91, State: Running, Role: LEADER
I20240502 07:59:31.276779 18860 consensus_queue.cc:237] T a53dba987d8d419c9b3c5ed35b02198f P d52c06abbcb44881a4837273c1ec8e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } }
I20240502 07:59:31.277990 18255 catalog_manager.cc:5484] T a53dba987d8d419c9b3c5ed35b02198f P d52c06abbcb44881a4837273c1ec8e91 reported cstate change: term changed from 0 to 1, leader changed from <none> to d52c06abbcb44881a4837273c1ec8e91 (127.3.139.65). New cstate: current_term: 1 leader_uuid: "d52c06abbcb44881a4837273c1ec8e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } health_report { overall_health: HEALTHY } } }
I20240502 07:59:31.435019 18914 raft_consensus.cc:482] T d1ee0467bf17461c9692fa887cf79cc2 P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:31.435161 18914 raft_consensus.cc:504] T d1ee0467bf17461c9692fa887cf79cc2 P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } }
I20240502 07:59:31.435482 18914 leader_election.cc:288] T d1ee0467bf17461c9692fa887cf79cc2 P dbe891b2f04e454db1c2b5de528e01c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d52c06abbcb44881a4837273c1ec8e91 (127.3.139.65:40139)
I20240502 07:59:31.435909 18396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1ee0467bf17461c9692fa887cf79cc2" candidate_uuid: "dbe891b2f04e454db1c2b5de528e01c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d52c06abbcb44881a4837273c1ec8e91" is_pre_election: true
I20240502 07:59:31.436084 18396 raft_consensus.cc:2454] T d1ee0467bf17461c9692fa887cf79cc2 P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe891b2f04e454db1c2b5de528e01c7 in term 0.
I20240502 07:59:31.436393 18729 leader_election.cc:302] T d1ee0467bf17461c9692fa887cf79cc2 P dbe891b2f04e454db1c2b5de528e01c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: d52c06abbcb44881a4837273c1ec8e91, dbe891b2f04e454db1c2b5de528e01c7; no voters:
I20240502 07:59:31.436570 18914 raft_consensus.cc:2781] T d1ee0467bf17461c9692fa887cf79cc2 P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:31.436631 18914 raft_consensus.cc:482] T d1ee0467bf17461c9692fa887cf79cc2 P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:31.436658 18914 raft_consensus.cc:3037] T d1ee0467bf17461c9692fa887cf79cc2 P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:31.437556 18914 raft_consensus.cc:504] T d1ee0467bf17461c9692fa887cf79cc2 P dbe891b2f04e454db1c2b5de528e01c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } }
I20240502 07:59:31.437810 18914 leader_election.cc:288] T d1ee0467bf17461c9692fa887cf79cc2 P dbe891b2f04e454db1c2b5de528e01c7 [CANDIDATE]: Term 1 election: Requested vote from peers d52c06abbcb44881a4837273c1ec8e91 (127.3.139.65:40139)
I20240502 07:59:31.438161 18396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1ee0467bf17461c9692fa887cf79cc2" candidate_uuid: "dbe891b2f04e454db1c2b5de528e01c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d52c06abbcb44881a4837273c1ec8e91"
I20240502 07:59:31.438266 18396 raft_consensus.cc:3037] T d1ee0467bf17461c9692fa887cf79cc2 P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:31.439020 18396 raft_consensus.cc:2454] T d1ee0467bf17461c9692fa887cf79cc2 P d52c06abbcb44881a4837273c1ec8e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe891b2f04e454db1c2b5de528e01c7 in term 1.
I20240502 07:59:31.439321 18729 leader_election.cc:302] T d1ee0467bf17461c9692fa887cf79cc2 P dbe891b2f04e454db1c2b5de528e01c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: d52c06abbcb44881a4837273c1ec8e91, dbe891b2f04e454db1c2b5de528e01c7; no voters:
I20240502 07:59:31.439460 18914 raft_consensus.cc:2781] T d1ee0467bf17461c9692fa887cf79cc2 P dbe891b2f04e454db1c2b5de528e01c7 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:31.439535 18914 raft_consensus.cc:686] T d1ee0467bf17461c9692fa887cf79cc2 P dbe891b2f04e454db1c2b5de528e01c7 [term 1 LEADER]: Becoming Leader. State: Replica: dbe891b2f04e454db1c2b5de528e01c7, State: Running, Role: LEADER
I20240502 07:59:31.439647 18914 consensus_queue.cc:237] T d1ee0467bf17461c9692fa887cf79cc2 P dbe891b2f04e454db1c2b5de528e01c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } }
I20240502 07:59:31.441068 18255 catalog_manager.cc:5484] T d1ee0467bf17461c9692fa887cf79cc2 P dbe891b2f04e454db1c2b5de528e01c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbe891b2f04e454db1c2b5de528e01c7 (127.3.139.68). New cstate: current_term: 1 leader_uuid: "dbe891b2f04e454db1c2b5de528e01c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } health_report { overall_health: UNKNOWN } } }
W20240502 07:59:31.570887 18927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:31.571105 18927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:31.573994 18927 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_1'
I20240502 07:59:31.638999 18793 raft_consensus.cc:1261] T bd37e09a008c4e85804e041122ea1c57 P dbe891b2f04e454db1c2b5de528e01c7 [term 1 FOLLOWER]: Refusing update from remote peer d52c06abbcb44881a4837273c1ec8e91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:31.639042 18792 raft_consensus.cc:1261] T a53dba987d8d419c9b3c5ed35b02198f P dbe891b2f04e454db1c2b5de528e01c7 [term 1 FOLLOWER]: Refusing update from remote peer d52c06abbcb44881a4837273c1ec8e91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:31.639832 18396 raft_consensus.cc:1261] T d1ee0467bf17461c9692fa887cf79cc2 P d52c06abbcb44881a4837273c1ec8e91 [term 1 FOLLOWER]: Refusing update from remote peer dbe891b2f04e454db1c2b5de528e01c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:31.639803 18860 consensus_queue.cc:1035] T a53dba987d8d419c9b3c5ed35b02198f P d52c06abbcb44881a4837273c1ec8e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:31.639951 18867 consensus_queue.cc:1035] T bd37e09a008c4e85804e041122ea1c57 P d52c06abbcb44881a4837273c1ec8e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:31.640264 18914 consensus_queue.cc:1035] T d1ee0467bf17461c9692fa887cf79cc2 P dbe891b2f04e454db1c2b5de528e01c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 74.2914 ms
time per row: 0.0371457 ms
I20240502 07:59:31.684032 18249 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51750:
name: "rebalance_test_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 2
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240502 07:59:31.684463 18249 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_2 in case of a server failure: 3 tablet servers would be needed, 2 are available. Consider bringing up more tablet servers.
I20240502 07:59:31.689034 18377 tablet_service.cc:1467] Processing CreateTablet for tablet 17b078447b4d49e0868219a0991b44c8 (DEFAULT_TABLE table=rebalance_test_table_2 [id=d0c3034cf9d8464bb3828569fba7b9e7]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:31.689292 18377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17b078447b4d49e0868219a0991b44c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:31.689386 18771 tablet_service.cc:1467] Processing CreateTablet for tablet 17b078447b4d49e0868219a0991b44c8 (DEFAULT_TABLE table=rebalance_test_table_2 [id=d0c3034cf9d8464bb3828569fba7b9e7]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:31.689612 18771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17b078447b4d49e0868219a0991b44c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:31.691322 18916 tablet_bootstrap.cc:492] T 17b078447b4d49e0868219a0991b44c8 P d52c06abbcb44881a4837273c1ec8e91: Bootstrap starting.
I20240502 07:59:31.691543 18376 tablet_service.cc:1467] Processing CreateTablet for tablet 0a10885caf66428880d2c862ca97f8db (DEFAULT_TABLE table=rebalance_test_table_2 [id=d0c3034cf9d8464bb3828569fba7b9e7]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:31.691764 18376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a10885caf66428880d2c862ca97f8db. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:31.692169 18916 tablet_bootstrap.cc:654] T 17b078447b4d49e0868219a0991b44c8 P d52c06abbcb44881a4837273c1ec8e91: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:31.693163 18918 tablet_bootstrap.cc:492] T 17b078447b4d49e0868219a0991b44c8 P dbe891b2f04e454db1c2b5de528e01c7: Bootstrap starting.
I20240502 07:59:31.693585 18916 tablet_bootstrap.cc:492] T 17b078447b4d49e0868219a0991b44c8 P d52c06abbcb44881a4837273c1ec8e91: No bootstrap required, opened a new log
I20240502 07:59:31.693651 18916 ts_tablet_manager.cc:1367] T 17b078447b4d49e0868219a0991b44c8 P d52c06abbcb44881a4837273c1ec8e91: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:31.693801 18918 tablet_bootstrap.cc:654] T 17b078447b4d49e0868219a0991b44c8 P dbe891b2f04e454db1c2b5de528e01c7: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:31.693997 18916 raft_consensus.cc:348] T 17b078447b4d49e0868219a0991b44c8 P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } }
I20240502 07:59:31.694085 18916 raft_consensus.cc:374] T 17b078447b4d49e0868219a0991b44c8 P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:31.694109 18916 raft_consensus.cc:729] T 17b078447b4d49e0868219a0991b44c8 P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d52c06abbcb44881a4837273c1ec8e91, State: Initialized, Role: FOLLOWER
I20240502 07:59:31.694207 18916 consensus_queue.cc:260] T 17b078447b4d49e0868219a0991b44c8 P d52c06abbcb44881a4837273c1ec8e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } }
I20240502 07:59:31.694430 18916 ts_tablet_manager.cc:1398] T 17b078447b4d49e0868219a0991b44c8 P d52c06abbcb44881a4837273c1ec8e91: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:31.694573 18916 tablet_bootstrap.cc:492] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91: Bootstrap starting.
I20240502 07:59:31.691540 18375 tablet_service.cc:1467] Processing CreateTablet for tablet 23442fad97424234ab659f3c0aecd0d0 (DEFAULT_TABLE table=rebalance_test_table_2 [id=d0c3034cf9d8464bb3828569fba7b9e7]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:31.695566 18375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23442fad97424234ab659f3c0aecd0d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:31.695516 18773 tablet_service.cc:1467] Processing CreateTablet for tablet 0a10885caf66428880d2c862ca97f8db (DEFAULT_TABLE table=rebalance_test_table_2 [id=d0c3034cf9d8464bb3828569fba7b9e7]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:31.695700 18773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a10885caf66428880d2c862ca97f8db. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:31.697996 18918 tablet_bootstrap.cc:492] T 17b078447b4d49e0868219a0991b44c8 P dbe891b2f04e454db1c2b5de528e01c7: No bootstrap required, opened a new log
I20240502 07:59:31.698066 18918 ts_tablet_manager.cc:1367] T 17b078447b4d49e0868219a0991b44c8 P dbe891b2f04e454db1c2b5de528e01c7: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 07:59:31.698427 18918 raft_consensus.cc:348] T 17b078447b4d49e0868219a0991b44c8 P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } }
I20240502 07:59:31.698523 18918 raft_consensus.cc:374] T 17b078447b4d49e0868219a0991b44c8 P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:31.698561 18918 raft_consensus.cc:729] T 17b078447b4d49e0868219a0991b44c8 P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbe891b2f04e454db1c2b5de528e01c7, State: Initialized, Role: FOLLOWER
I20240502 07:59:31.698671 18918 consensus_queue.cc:260] T 17b078447b4d49e0868219a0991b44c8 P dbe891b2f04e454db1c2b5de528e01c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } }
I20240502 07:59:31.698915 18918 ts_tablet_manager.cc:1398] T 17b078447b4d49e0868219a0991b44c8 P dbe891b2f04e454db1c2b5de528e01c7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:31.699067 18918 tablet_bootstrap.cc:492] T 0a10885caf66428880d2c862ca97f8db P dbe891b2f04e454db1c2b5de528e01c7: Bootstrap starting.
I20240502 07:59:31.699607 18918 tablet_bootstrap.cc:654] T 0a10885caf66428880d2c862ca97f8db P dbe891b2f04e454db1c2b5de528e01c7: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:31.700026 18771 tablet_service.cc:1467] Processing CreateTablet for tablet 23442fad97424234ab659f3c0aecd0d0 (DEFAULT_TABLE table=rebalance_test_table_2 [id=d0c3034cf9d8464bb3828569fba7b9e7]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:31.700249 18771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23442fad97424234ab659f3c0aecd0d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:31.701813 18918 tablet_bootstrap.cc:492] T 0a10885caf66428880d2c862ca97f8db P dbe891b2f04e454db1c2b5de528e01c7: No bootstrap required, opened a new log
I20240502 07:59:31.695363 18916 tablet_bootstrap.cc:654] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:31.701973 18918 ts_tablet_manager.cc:1367] T 0a10885caf66428880d2c862ca97f8db P dbe891b2f04e454db1c2b5de528e01c7: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:59:31.702293 18918 raft_consensus.cc:348] T 0a10885caf66428880d2c862ca97f8db P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } }
I20240502 07:59:31.702355 18918 raft_consensus.cc:374] T 0a10885caf66428880d2c862ca97f8db P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:31.702373 18918 raft_consensus.cc:729] T 0a10885caf66428880d2c862ca97f8db P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbe891b2f04e454db1c2b5de528e01c7, State: Initialized, Role: FOLLOWER
I20240502 07:59:31.702459 18918 consensus_queue.cc:260] T 0a10885caf66428880d2c862ca97f8db P dbe891b2f04e454db1c2b5de528e01c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } }
I20240502 07:59:31.702730 18918 ts_tablet_manager.cc:1398] T 0a10885caf66428880d2c862ca97f8db P dbe891b2f04e454db1c2b5de528e01c7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:31.702883 18918 tablet_bootstrap.cc:492] T 23442fad97424234ab659f3c0aecd0d0 P dbe891b2f04e454db1c2b5de528e01c7: Bootstrap starting.
I20240502 07:59:31.702950 18916 tablet_bootstrap.cc:492] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91: No bootstrap required, opened a new log
I20240502 07:59:31.703028 18916 ts_tablet_manager.cc:1367] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240502 07:59:31.703438 18916 raft_consensus.cc:348] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } }
I20240502 07:59:31.703605 18916 raft_consensus.cc:374] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:31.703684 18916 raft_consensus.cc:729] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d52c06abbcb44881a4837273c1ec8e91, State: Initialized, Role: FOLLOWER
I20240502 07:59:31.703850 18916 consensus_queue.cc:260] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } }
I20240502 07:59:31.703955 18918 tablet_bootstrap.cc:654] T 23442fad97424234ab659f3c0aecd0d0 P dbe891b2f04e454db1c2b5de528e01c7: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:31.704164 18916 ts_tablet_manager.cc:1398] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:31.704332 18916 tablet_bootstrap.cc:492] T 23442fad97424234ab659f3c0aecd0d0 P d52c06abbcb44881a4837273c1ec8e91: Bootstrap starting.
I20240502 07:59:31.705183 18916 tablet_bootstrap.cc:654] T 23442fad97424234ab659f3c0aecd0d0 P d52c06abbcb44881a4837273c1ec8e91: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:31.705229 18918 tablet_bootstrap.cc:492] T 23442fad97424234ab659f3c0aecd0d0 P dbe891b2f04e454db1c2b5de528e01c7: No bootstrap required, opened a new log
I20240502 07:59:31.705303 18918 ts_tablet_manager.cc:1367] T 23442fad97424234ab659f3c0aecd0d0 P dbe891b2f04e454db1c2b5de528e01c7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:31.705577 18918 raft_consensus.cc:348] T 23442fad97424234ab659f3c0aecd0d0 P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } }
I20240502 07:59:31.705634 18918 raft_consensus.cc:374] T 23442fad97424234ab659f3c0aecd0d0 P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:31.705651 18918 raft_consensus.cc:729] T 23442fad97424234ab659f3c0aecd0d0 P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbe891b2f04e454db1c2b5de528e01c7, State: Initialized, Role: FOLLOWER
I20240502 07:59:31.705737 18918 consensus_queue.cc:260] T 23442fad97424234ab659f3c0aecd0d0 P dbe891b2f04e454db1c2b5de528e01c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } }
I20240502 07:59:31.705940 18918 ts_tablet_manager.cc:1398] T 23442fad97424234ab659f3c0aecd0d0 P dbe891b2f04e454db1c2b5de528e01c7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:31.706008 18916 tablet_bootstrap.cc:492] T 23442fad97424234ab659f3c0aecd0d0 P d52c06abbcb44881a4837273c1ec8e91: No bootstrap required, opened a new log
I20240502 07:59:31.706064 18916 ts_tablet_manager.cc:1367] T 23442fad97424234ab659f3c0aecd0d0 P d52c06abbcb44881a4837273c1ec8e91: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:31.706398 18916 raft_consensus.cc:348] T 23442fad97424234ab659f3c0aecd0d0 P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } }
I20240502 07:59:31.706487 18916 raft_consensus.cc:374] T 23442fad97424234ab659f3c0aecd0d0 P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:31.706547 18916 raft_consensus.cc:729] T 23442fad97424234ab659f3c0aecd0d0 P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d52c06abbcb44881a4837273c1ec8e91, State: Initialized, Role: FOLLOWER
I20240502 07:59:31.706676 18916 consensus_queue.cc:260] T 23442fad97424234ab659f3c0aecd0d0 P d52c06abbcb44881a4837273c1ec8e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } }
I20240502 07:59:31.706948 18916 ts_tablet_manager.cc:1398] T 23442fad97424234ab659f3c0aecd0d0 P d52c06abbcb44881a4837273c1ec8e91: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:31.734591 18867 raft_consensus.cc:482] T 17b078447b4d49e0868219a0991b44c8 P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:31.734695 18867 raft_consensus.cc:504] T 17b078447b4d49e0868219a0991b44c8 P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } }
I20240502 07:59:31.734953 18867 leader_election.cc:288] T 17b078447b4d49e0868219a0991b44c8 P d52c06abbcb44881a4837273c1ec8e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dbe891b2f04e454db1c2b5de528e01c7 (127.3.139.68:43983)
I20240502 07:59:31.735271 18792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17b078447b4d49e0868219a0991b44c8" candidate_uuid: "d52c06abbcb44881a4837273c1ec8e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbe891b2f04e454db1c2b5de528e01c7" is_pre_election: true
I20240502 07:59:31.735373 18792 raft_consensus.cc:2454] T 17b078447b4d49e0868219a0991b44c8 P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d52c06abbcb44881a4837273c1ec8e91 in term 0.
I20240502 07:59:31.735632 18332 leader_election.cc:302] T 17b078447b4d49e0868219a0991b44c8 P d52c06abbcb44881a4837273c1ec8e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: d52c06abbcb44881a4837273c1ec8e91, dbe891b2f04e454db1c2b5de528e01c7; no voters:
I20240502 07:59:31.735777 18867 raft_consensus.cc:2781] T 17b078447b4d49e0868219a0991b44c8 P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:31.735831 18867 raft_consensus.cc:482] T 17b078447b4d49e0868219a0991b44c8 P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:31.735874 18867 raft_consensus.cc:3037] T 17b078447b4d49e0868219a0991b44c8 P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:31.736637 18867 raft_consensus.cc:504] T 17b078447b4d49e0868219a0991b44c8 P d52c06abbcb44881a4837273c1ec8e91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } }
I20240502 07:59:31.736864 18867 leader_election.cc:288] T 17b078447b4d49e0868219a0991b44c8 P d52c06abbcb44881a4837273c1ec8e91 [CANDIDATE]: Term 1 election: Requested vote from peers dbe891b2f04e454db1c2b5de528e01c7 (127.3.139.68:43983)
I20240502 07:59:31.737192 18792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17b078447b4d49e0868219a0991b44c8" candidate_uuid: "d52c06abbcb44881a4837273c1ec8e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbe891b2f04e454db1c2b5de528e01c7"
I20240502 07:59:31.737291 18792 raft_consensus.cc:3037] T 17b078447b4d49e0868219a0991b44c8 P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:31.737941 18792 raft_consensus.cc:2454] T 17b078447b4d49e0868219a0991b44c8 P dbe891b2f04e454db1c2b5de528e01c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d52c06abbcb44881a4837273c1ec8e91 in term 1.
I20240502 07:59:31.738166 18332 leader_election.cc:302] T 17b078447b4d49e0868219a0991b44c8 P d52c06abbcb44881a4837273c1ec8e91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: d52c06abbcb44881a4837273c1ec8e91, dbe891b2f04e454db1c2b5de528e01c7; no voters:
I20240502 07:59:31.738307 18867 raft_consensus.cc:2781] T 17b078447b4d49e0868219a0991b44c8 P d52c06abbcb44881a4837273c1ec8e91 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:31.738389 18867 raft_consensus.cc:686] T 17b078447b4d49e0868219a0991b44c8 P d52c06abbcb44881a4837273c1ec8e91 [term 1 LEADER]: Becoming Leader. State: Replica: d52c06abbcb44881a4837273c1ec8e91, State: Running, Role: LEADER
I20240502 07:59:31.738507 18867 consensus_queue.cc:237] T 17b078447b4d49e0868219a0991b44c8 P d52c06abbcb44881a4837273c1ec8e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } }
I20240502 07:59:31.739779 18250 catalog_manager.cc:5484] T 17b078447b4d49e0868219a0991b44c8 P d52c06abbcb44881a4837273c1ec8e91 reported cstate change: term changed from 0 to 1, leader changed from <none> to d52c06abbcb44881a4837273c1ec8e91 (127.3.139.65). New cstate: current_term: 1 leader_uuid: "d52c06abbcb44881a4837273c1ec8e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } health_report { overall_health: HEALTHY } } }
I20240502 07:59:31.791811 18867 raft_consensus.cc:482] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:31.791889 18867 raft_consensus.cc:504] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } }
I20240502 07:59:31.792129 18867 leader_election.cc:288] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dbe891b2f04e454db1c2b5de528e01c7 (127.3.139.68:43983)
I20240502 07:59:31.792423 18792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a10885caf66428880d2c862ca97f8db" candidate_uuid: "d52c06abbcb44881a4837273c1ec8e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbe891b2f04e454db1c2b5de528e01c7" is_pre_election: true
I20240502 07:59:31.792528 18792 raft_consensus.cc:2454] T 0a10885caf66428880d2c862ca97f8db P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d52c06abbcb44881a4837273c1ec8e91 in term 0.
I20240502 07:59:31.792762 18332 leader_election.cc:302] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: d52c06abbcb44881a4837273c1ec8e91, dbe891b2f04e454db1c2b5de528e01c7; no voters:
I20240502 07:59:31.792908 18867 raft_consensus.cc:2781] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:31.792963 18867 raft_consensus.cc:482] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:31.792999 18867 raft_consensus.cc:3037] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:31.793624 18867 raft_consensus.cc:504] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } }
I20240502 07:59:31.793841 18867 leader_election.cc:288] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91 [CANDIDATE]: Term 1 election: Requested vote from peers dbe891b2f04e454db1c2b5de528e01c7 (127.3.139.68:43983)
I20240502 07:59:31.794100 18792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a10885caf66428880d2c862ca97f8db" candidate_uuid: "d52c06abbcb44881a4837273c1ec8e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbe891b2f04e454db1c2b5de528e01c7"
I20240502 07:59:31.794203 18792 raft_consensus.cc:3037] T 0a10885caf66428880d2c862ca97f8db P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:31.794986 18792 raft_consensus.cc:2454] T 0a10885caf66428880d2c862ca97f8db P dbe891b2f04e454db1c2b5de528e01c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d52c06abbcb44881a4837273c1ec8e91 in term 1.
I20240502 07:59:31.795267 18332 leader_election.cc:302] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: d52c06abbcb44881a4837273c1ec8e91, dbe891b2f04e454db1c2b5de528e01c7; no voters:
I20240502 07:59:31.795410 18867 raft_consensus.cc:2781] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:31.795486 18867 raft_consensus.cc:686] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91 [term 1 LEADER]: Becoming Leader. State: Replica: d52c06abbcb44881a4837273c1ec8e91, State: Running, Role: LEADER
I20240502 07:59:31.795590 18867 consensus_queue.cc:237] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } }
I20240502 07:59:31.796799 18250 catalog_manager.cc:5484] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91 reported cstate change: term changed from 0 to 1, leader changed from <none> to d52c06abbcb44881a4837273c1ec8e91 (127.3.139.65). New cstate: current_term: 1 leader_uuid: "d52c06abbcb44881a4837273c1ec8e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } health_report { overall_health: HEALTHY } } }
I20240502 07:59:31.947443 18867 raft_consensus.cc:482] T 23442fad97424234ab659f3c0aecd0d0 P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:31.947551 18867 raft_consensus.cc:504] T 23442fad97424234ab659f3c0aecd0d0 P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } }
I20240502 07:59:31.947846 18867 leader_election.cc:288] T 23442fad97424234ab659f3c0aecd0d0 P d52c06abbcb44881a4837273c1ec8e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dbe891b2f04e454db1c2b5de528e01c7 (127.3.139.68:43983)
I20240502 07:59:31.948230 18792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23442fad97424234ab659f3c0aecd0d0" candidate_uuid: "d52c06abbcb44881a4837273c1ec8e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbe891b2f04e454db1c2b5de528e01c7" is_pre_election: true
I20240502 07:59:31.948348 18792 raft_consensus.cc:2454] T 23442fad97424234ab659f3c0aecd0d0 P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d52c06abbcb44881a4837273c1ec8e91 in term 0.
I20240502 07:59:31.948614 18332 leader_election.cc:302] T 23442fad97424234ab659f3c0aecd0d0 P d52c06abbcb44881a4837273c1ec8e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: d52c06abbcb44881a4837273c1ec8e91, dbe891b2f04e454db1c2b5de528e01c7; no voters:
I20240502 07:59:31.948768 18867 raft_consensus.cc:2781] T 23442fad97424234ab659f3c0aecd0d0 P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:31.948825 18867 raft_consensus.cc:482] T 23442fad97424234ab659f3c0aecd0d0 P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:31.948863 18867 raft_consensus.cc:3037] T 23442fad97424234ab659f3c0aecd0d0 P d52c06abbcb44881a4837273c1ec8e91 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:31.949592 18867 raft_consensus.cc:504] T 23442fad97424234ab659f3c0aecd0d0 P d52c06abbcb44881a4837273c1ec8e91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } }
I20240502 07:59:31.949847 18867 leader_election.cc:288] T 23442fad97424234ab659f3c0aecd0d0 P d52c06abbcb44881a4837273c1ec8e91 [CANDIDATE]: Term 1 election: Requested vote from peers dbe891b2f04e454db1c2b5de528e01c7 (127.3.139.68:43983)
I20240502 07:59:31.950170 18792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23442fad97424234ab659f3c0aecd0d0" candidate_uuid: "d52c06abbcb44881a4837273c1ec8e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbe891b2f04e454db1c2b5de528e01c7"
I20240502 07:59:31.950263 18792 raft_consensus.cc:3037] T 23442fad97424234ab659f3c0aecd0d0 P dbe891b2f04e454db1c2b5de528e01c7 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:31.950929 18792 raft_consensus.cc:2454] T 23442fad97424234ab659f3c0aecd0d0 P dbe891b2f04e454db1c2b5de528e01c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d52c06abbcb44881a4837273c1ec8e91 in term 1.
I20240502 07:59:31.951181 18332 leader_election.cc:302] T 23442fad97424234ab659f3c0aecd0d0 P d52c06abbcb44881a4837273c1ec8e91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: d52c06abbcb44881a4837273c1ec8e91, dbe891b2f04e454db1c2b5de528e01c7; no voters:
I20240502 07:59:31.951320 18867 raft_consensus.cc:2781] T 23442fad97424234ab659f3c0aecd0d0 P d52c06abbcb44881a4837273c1ec8e91 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:31.951404 18867 raft_consensus.cc:686] T 23442fad97424234ab659f3c0aecd0d0 P d52c06abbcb44881a4837273c1ec8e91 [term 1 LEADER]: Becoming Leader. State: Replica: d52c06abbcb44881a4837273c1ec8e91, State: Running, Role: LEADER
I20240502 07:59:31.951525 18867 consensus_queue.cc:237] T 23442fad97424234ab659f3c0aecd0d0 P d52c06abbcb44881a4837273c1ec8e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } }
I20240502 07:59:31.952908 18250 catalog_manager.cc:5484] T 23442fad97424234ab659f3c0aecd0d0 P d52c06abbcb44881a4837273c1ec8e91 reported cstate change: term changed from 0 to 1, leader changed from <none> to d52c06abbcb44881a4837273c1ec8e91 (127.3.139.65). New cstate: current_term: 1 leader_uuid: "d52c06abbcb44881a4837273c1ec8e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } health_report { overall_health: UNKNOWN } } }
W20240502 07:59:32.077558 18968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:32.077771 18968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:32.080734 18968 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_2'
I20240502 07:59:32.163389 18792 raft_consensus.cc:1261] T 0a10885caf66428880d2c862ca97f8db P dbe891b2f04e454db1c2b5de528e01c7 [term 1 FOLLOWER]: Refusing update from remote peer d52c06abbcb44881a4837273c1ec8e91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:32.164290 18793 raft_consensus.cc:1261] T 17b078447b4d49e0868219a0991b44c8 P dbe891b2f04e454db1c2b5de528e01c7 [term 1 FOLLOWER]: Refusing update from remote peer d52c06abbcb44881a4837273c1ec8e91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:32.164693 18867 consensus_queue.cc:1035] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:32.165272 18792 raft_consensus.cc:1261] T 23442fad97424234ab659f3c0aecd0d0 P dbe891b2f04e454db1c2b5de528e01c7 [term 1 FOLLOWER]: Refusing update from remote peer d52c06abbcb44881a4837273c1ec8e91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:32.166023 18860 consensus_queue.cc:1035] T 17b078447b4d49e0868219a0991b44c8 P d52c06abbcb44881a4837273c1ec8e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:32.166265 18860 consensus_queue.cc:1035] T 23442fad97424234ab659f3c0aecd0d0 P d52c06abbcb44881a4837273c1ec8e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 92.9683 ms
time per row: 0.0464841 ms
I20240502 07:59:32.208997 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.66:35635
--local_ip_for_outbound_sockets=127.3.139.66
--tserver_master_addrs=127.3.139.126:43361
--webserver_port=34827
--webserver_interface=127.3.139.66
--builtin_ntp_servers=127.3.139.84:44343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240502 07:59:32.310216 19007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:32.310402 19007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:59:32.310441 19007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:32.313313 19007 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:59:32.313371 19007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:59:32.313432 19007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.66
I20240502 07:59:32.316828 19007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:44343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.66:35635
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.3.139.66
--webserver_port=34827
--tserver_master_addrs=127.3.139.126:43361
--never_fsync=true
--heap_profile_path=/tmp/kudu.19007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.66
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:59:32.317526 19007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:59:32.318480 19007 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:59:32.326045 19015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:32.326355 19012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:32.326821 19013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:32.327257 19007 server_base.cc:1034] running on GCE node
I20240502 07:59:32.327596 19007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:59:32.328073 19007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:59:32.329205 19007 hybrid_clock.cc:648] HybridClock initialized: now 1714636772329182 us; error 39 us; skew 500 ppm
I20240502 07:59:32.330737 19007 webserver.cc:460] Webserver started at http://127.3.139.66:34827/ using document root <none> and password file <none>
I20240502 07:59:32.331246 19007 fs_manager.cc:359] Metadata directory not provided
I20240502 07:59:32.331310 19007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:59:32.334236 19007 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:32.335887 19021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:32.336835 19007 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:32.336922 19007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "6fd33d34de9c463183177f1a13d4f851"
format_stamp: "Formatted at 2024-05-02 07:59:26 on dist-test-slave-twcr"
I20240502 07:59:32.337250 19007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:59:32.361335 19007 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:59:32.361835 19007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:59:32.361958 19007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:59:32.362393 19007 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:59:32.363212 19007 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:59:32.363255 19007 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:32.363289 19007 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:59:32.363309 19007 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:32.370682 19007 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.66:35635
I20240502 07:59:32.370713 19134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.66:35635 every 8 connection(s)
I20240502 07:59:32.371600 19007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240502 07:59:32.374161 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 19007
I20240502 07:59:32.374532 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.67:33695
--local_ip_for_outbound_sockets=127.3.139.67
--tserver_master_addrs=127.3.139.126:43361
--webserver_port=46239
--webserver_interface=127.3.139.67
--builtin_ntp_servers=127.3.139.84:44343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 07:59:32.379774 19135 heartbeater.cc:347] Connected to a master server at 127.3.139.126:43361
I20240502 07:59:32.379961 19135 heartbeater.cc:464] Registering TS with master...
I20240502 07:59:32.380394 19135 heartbeater.cc:511] Master 127.3.139.126:43361 requested a full tablet report, sending...
I20240502 07:59:32.381539 18249 ts_manager.cc:194] Re-registered known tserver with Master: 6fd33d34de9c463183177f1a13d4f851 (127.3.139.66:35635)
I20240502 07:59:32.382272 18249 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.66:36955
W20240502 07:59:32.474001 19138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:32.474170 19138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:59:32.474211 19138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:32.477149 19138 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:59:32.477209 19138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:59:32.477270 19138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.67
I20240502 07:59:32.480779 19138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:44343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.67:33695
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.3.139.67
--webserver_port=46239
--tserver_master_addrs=127.3.139.126:43361
--never_fsync=true
--heap_profile_path=/tmp/kudu.19138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.67
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:59:32.481516 19138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:59:32.482513 19138 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:59:32.490028 19146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:32.490396 19143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:32.490731 19144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:32.491221 19138 server_base.cc:1034] running on GCE node
I20240502 07:59:32.491542 19138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:59:32.492007 19138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:59:32.493151 19138 hybrid_clock.cc:648] HybridClock initialized: now 1714636772493130 us; error 32 us; skew 500 ppm
I20240502 07:59:32.494772 19138 webserver.cc:460] Webserver started at http://127.3.139.67:46239/ using document root <none> and password file <none>
I20240502 07:59:32.495311 19138 fs_manager.cc:359] Metadata directory not provided
I20240502 07:59:32.495376 19138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:59:32.498514 19138 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240502 07:59:32.500329 19152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:32.501226 19138 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:32.501346 19138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "db667da4313e43e2917b5acc7e6f645d"
format_stamp: "Formatted at 2024-05-02 07:59:26 on dist-test-slave-twcr"
I20240502 07:59:32.501720 19138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:59:32.514456 19138 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:59:32.514935 19138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:59:32.515090 19138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:59:32.515544 19138 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:59:32.516347 19138 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:59:32.516392 19138 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:32.516429 19138 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:59:32.516458 19138 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:32.524258 19138 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.67:33695
I20240502 07:59:32.524304 19265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.67:33695 every 8 connection(s)
I20240502 07:59:32.525136 19138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240502 07:59:32.530000 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 19138
I20240502 07:59:32.532958 19266 heartbeater.cc:347] Connected to a master server at 127.3.139.126:43361
I20240502 07:59:32.533149 19266 heartbeater.cc:464] Registering TS with master...
I20240502 07:59:32.533603 19266 heartbeater.cc:511] Master 127.3.139.126:43361 requested a full tablet report, sending...
I20240502 07:59:32.534479 18249 ts_manager.cc:194] Re-registered known tserver with Master: db667da4313e43e2917b5acc7e6f645d (127.3.139.67:33695)
I20240502 07:59:32.534991 18249 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.67:37653
I20240502 07:59:32.704757 19069 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:32.705618 18771 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:59:32.705618 18375 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:32.706343 19200 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:32.787194 19069 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:32.788237 18375 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:32.788609 18771 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:59:32.789697 19200 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:32.889919 19069 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:32.890161 19200 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:32.890295 18375 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:32.890667 18771 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:59:32.968777 18375 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:32.969022 19069 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:32.973587 18771 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:59:32.985982 19200 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:33.011739 18396 consensus_queue.cc:237] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } attrs { replace: true } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: NON_VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: true } }
I20240502 07:59:33.013144 18790 raft_consensus.cc:1261] T 0a10885caf66428880d2c862ca97f8db P dbe891b2f04e454db1c2b5de528e01c7 [term 1 FOLLOWER]: Refusing update from remote peer d52c06abbcb44881a4837273c1ec8e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:33.013540 19407 consensus_queue.cc:1035] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:33.015295 18860 raft_consensus.cc:2932] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 6fd33d34de9c463183177f1a13d4f851 (127.3.139.66) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } attrs { replace: true } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: NON_VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: true } } }
I20240502 07:59:33.015725 18790 raft_consensus.cc:2932] T 0a10885caf66428880d2c862ca97f8db P dbe891b2f04e454db1c2b5de528e01c7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 6fd33d34de9c463183177f1a13d4f851 (127.3.139.66) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } attrs { replace: true } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: NON_VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: true } } }
W20240502 07:59:33.016609 18331 consensus_peers.cc:485] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91 -> Peer 6fd33d34de9c463183177f1a13d4f851 (127.3.139.66:35635): Couldn't send request to peer 6fd33d34de9c463183177f1a13d4f851. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0a10885caf66428880d2c862ca97f8db. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:33.018162 18249 catalog_manager.cc:5484] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91 reported cstate change: config changed from index -1 to 4, NON_VOTER 6fd33d34de9c463183177f1a13d4f851 (127.3.139.66) added. New cstate: current_term: 1 leader_uuid: "d52c06abbcb44881a4837273c1ec8e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: NON_VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:33.024327 18396 consensus_queue.cc:237] T 58aa0644416e4e698766581b577d21d0 P d52c06abbcb44881a4837273c1ec8e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } attrs { replace: true } } peers { permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: NON_VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: true } }
I20240502 07:59:33.025537 18790 raft_consensus.cc:1261] T 58aa0644416e4e698766581b577d21d0 P dbe891b2f04e454db1c2b5de528e01c7 [term 1 FOLLOWER]: Refusing update from remote peer d52c06abbcb44881a4837273c1ec8e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:33.025876 19409 consensus_queue.cc:1035] T 58aa0644416e4e698766581b577d21d0 P d52c06abbcb44881a4837273c1ec8e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:33.027428 19409 raft_consensus.cc:2932] T 58aa0644416e4e698766581b577d21d0 P d52c06abbcb44881a4837273c1ec8e91 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER db667da4313e43e2917b5acc7e6f645d (127.3.139.67) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } attrs { replace: true } } peers { permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: NON_VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: true } } }
I20240502 07:59:33.027724 18790 raft_consensus.cc:2932] T 58aa0644416e4e698766581b577d21d0 P dbe891b2f04e454db1c2b5de528e01c7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER db667da4313e43e2917b5acc7e6f645d (127.3.139.67) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } attrs { replace: true } } peers { permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: NON_VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: true } } }
W20240502 07:59:33.029464 18333 consensus_peers.cc:485] T 58aa0644416e4e698766581b577d21d0 P d52c06abbcb44881a4837273c1ec8e91 -> Peer db667da4313e43e2917b5acc7e6f645d (127.3.139.67:33695): Couldn't send request to peer db667da4313e43e2917b5acc7e6f645d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 58aa0644416e4e698766581b577d21d0. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:33.029502 18250 catalog_manager.cc:5484] T 58aa0644416e4e698766581b577d21d0 P d52c06abbcb44881a4837273c1ec8e91 reported cstate change: config changed from index -1 to 4, NON_VOTER db667da4313e43e2917b5acc7e6f645d (127.3.139.67) added. New cstate: current_term: 1 leader_uuid: "d52c06abbcb44881a4837273c1ec8e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: NON_VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:33.036365 18790 consensus_queue.cc:237] T d1ee0467bf17461c9692fa887cf79cc2 P dbe891b2f04e454db1c2b5de528e01c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } attrs { replace: true } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: NON_VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: true } }
I20240502 07:59:33.037645 18396 raft_consensus.cc:1261] T d1ee0467bf17461c9692fa887cf79cc2 P d52c06abbcb44881a4837273c1ec8e91 [term 1 FOLLOWER]: Refusing update from remote peer dbe891b2f04e454db1c2b5de528e01c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:33.038085 19427 consensus_queue.cc:1035] T d1ee0467bf17461c9692fa887cf79cc2 P dbe891b2f04e454db1c2b5de528e01c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:33.039876 18870 raft_consensus.cc:2932] T d1ee0467bf17461c9692fa887cf79cc2 P dbe891b2f04e454db1c2b5de528e01c7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 6fd33d34de9c463183177f1a13d4f851 (127.3.139.66) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } attrs { replace: true } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: NON_VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: true } } }
I20240502 07:59:33.040262 18396 raft_consensus.cc:2932] T d1ee0467bf17461c9692fa887cf79cc2 P d52c06abbcb44881a4837273c1ec8e91 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 6fd33d34de9c463183177f1a13d4f851 (127.3.139.66) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } attrs { replace: true } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: NON_VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: true } } }
W20240502 07:59:33.042290 18727 consensus_peers.cc:485] T d1ee0467bf17461c9692fa887cf79cc2 P dbe891b2f04e454db1c2b5de528e01c7 -> Peer 6fd33d34de9c463183177f1a13d4f851 (127.3.139.66:35635): Couldn't send request to peer 6fd33d34de9c463183177f1a13d4f851. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d1ee0467bf17461c9692fa887cf79cc2. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:33.042526 18249 catalog_manager.cc:5484] T d1ee0467bf17461c9692fa887cf79cc2 P dbe891b2f04e454db1c2b5de528e01c7 reported cstate change: config changed from index -1 to 4, NON_VOTER 6fd33d34de9c463183177f1a13d4f851 (127.3.139.66) added. New cstate: current_term: 1 leader_uuid: "dbe891b2f04e454db1c2b5de528e01c7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: NON_VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:33.048749 18396 consensus_queue.cc:237] T bd37e09a008c4e85804e041122ea1c57 P d52c06abbcb44881a4837273c1ec8e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } attrs { replace: true } } peers { permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: NON_VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: true } }
I20240502 07:59:33.049762 18790 raft_consensus.cc:1261] T bd37e09a008c4e85804e041122ea1c57 P dbe891b2f04e454db1c2b5de528e01c7 [term 1 FOLLOWER]: Refusing update from remote peer d52c06abbcb44881a4837273c1ec8e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 07:59:33.049932 18333 consensus_peers.cc:485] T bd37e09a008c4e85804e041122ea1c57 P d52c06abbcb44881a4837273c1ec8e91 -> Peer db667da4313e43e2917b5acc7e6f645d (127.3.139.67:33695): Couldn't send request to peer db667da4313e43e2917b5acc7e6f645d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bd37e09a008c4e85804e041122ea1c57. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:33.050153 19409 consensus_queue.cc:1035] T bd37e09a008c4e85804e041122ea1c57 P d52c06abbcb44881a4837273c1ec8e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:33.051623 19407 raft_consensus.cc:2932] T bd37e09a008c4e85804e041122ea1c57 P d52c06abbcb44881a4837273c1ec8e91 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER db667da4313e43e2917b5acc7e6f645d (127.3.139.67) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } attrs { replace: true } } peers { permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: NON_VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: true } } }
I20240502 07:59:33.052006 18790 raft_consensus.cc:2932] T bd37e09a008c4e85804e041122ea1c57 P dbe891b2f04e454db1c2b5de528e01c7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER db667da4313e43e2917b5acc7e6f645d (127.3.139.67) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } attrs { replace: true } } peers { permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: NON_VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: true } } }
I20240502 07:59:33.053999 18249 catalog_manager.cc:5484] T bd37e09a008c4e85804e041122ea1c57 P d52c06abbcb44881a4837273c1ec8e91 reported cstate change: config changed from index -1 to 4, NON_VOTER db667da4313e43e2917b5acc7e6f645d (127.3.139.67) added. New cstate: current_term: 1 leader_uuid: "d52c06abbcb44881a4837273c1ec8e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: NON_VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:33.059490 18790 consensus_queue.cc:237] T d8c026e315ac4622b77b618de5c0b09f P dbe891b2f04e454db1c2b5de528e01c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } attrs { replace: true } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: NON_VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: true } }
I20240502 07:59:33.060678 18396 raft_consensus.cc:1261] T d8c026e315ac4622b77b618de5c0b09f P d52c06abbcb44881a4837273c1ec8e91 [term 1 FOLLOWER]: Refusing update from remote peer dbe891b2f04e454db1c2b5de528e01c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 07:59:33.060688 18727 consensus_peers.cc:485] T d8c026e315ac4622b77b618de5c0b09f P dbe891b2f04e454db1c2b5de528e01c7 -> Peer 6fd33d34de9c463183177f1a13d4f851 (127.3.139.66:35635): Couldn't send request to peer 6fd33d34de9c463183177f1a13d4f851. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d8c026e315ac4622b77b618de5c0b09f. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:33.060979 19427 consensus_queue.cc:1035] T d8c026e315ac4622b77b618de5c0b09f P dbe891b2f04e454db1c2b5de528e01c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:33.062362 19431 raft_consensus.cc:2932] T d8c026e315ac4622b77b618de5c0b09f P dbe891b2f04e454db1c2b5de528e01c7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 6fd33d34de9c463183177f1a13d4f851 (127.3.139.66) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } attrs { replace: true } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: NON_VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: true } } }
I20240502 07:59:33.062791 18396 raft_consensus.cc:2932] T d8c026e315ac4622b77b618de5c0b09f P d52c06abbcb44881a4837273c1ec8e91 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 6fd33d34de9c463183177f1a13d4f851 (127.3.139.66) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } attrs { replace: true } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: NON_VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: true } } }
I20240502 07:59:33.064327 18250 catalog_manager.cc:5484] T d8c026e315ac4622b77b618de5c0b09f P dbe891b2f04e454db1c2b5de528e01c7 reported cstate change: config changed from index -1 to 4, NON_VOTER 6fd33d34de9c463183177f1a13d4f851 (127.3.139.66) added. New cstate: current_term: 1 leader_uuid: "dbe891b2f04e454db1c2b5de528e01c7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: NON_VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:33.070660 18396 consensus_queue.cc:237] T 23442fad97424234ab659f3c0aecd0d0 P d52c06abbcb44881a4837273c1ec8e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } attrs { replace: true } } peers { permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: NON_VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: true } }
I20240502 07:59:33.071650 18790 raft_consensus.cc:1261] T 23442fad97424234ab659f3c0aecd0d0 P dbe891b2f04e454db1c2b5de528e01c7 [term 1 FOLLOWER]: Refusing update from remote peer d52c06abbcb44881a4837273c1ec8e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 07:59:33.071815 18333 consensus_peers.cc:485] T 23442fad97424234ab659f3c0aecd0d0 P d52c06abbcb44881a4837273c1ec8e91 -> Peer db667da4313e43e2917b5acc7e6f645d (127.3.139.67:33695): Couldn't send request to peer db667da4313e43e2917b5acc7e6f645d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 23442fad97424234ab659f3c0aecd0d0. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:33.071938 19409 consensus_queue.cc:1035] T 23442fad97424234ab659f3c0aecd0d0 P d52c06abbcb44881a4837273c1ec8e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:33.073194 19407 raft_consensus.cc:2932] T 23442fad97424234ab659f3c0aecd0d0 P d52c06abbcb44881a4837273c1ec8e91 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER db667da4313e43e2917b5acc7e6f645d (127.3.139.67) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } attrs { replace: true } } peers { permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: NON_VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: true } } }
I20240502 07:59:33.073621 18790 raft_consensus.cc:2932] T 23442fad97424234ab659f3c0aecd0d0 P dbe891b2f04e454db1c2b5de528e01c7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER db667da4313e43e2917b5acc7e6f645d (127.3.139.67) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } attrs { replace: true } } peers { permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: NON_VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: true } } }
I20240502 07:59:33.075569 18250 catalog_manager.cc:5484] T 23442fad97424234ab659f3c0aecd0d0 P d52c06abbcb44881a4837273c1ec8e91 reported cstate change: config changed from index -1 to 4, NON_VOTER db667da4313e43e2917b5acc7e6f645d (127.3.139.67) added. New cstate: current_term: 1 leader_uuid: "d52c06abbcb44881a4837273c1ec8e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: NON_VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:33.384037 19135 heartbeater.cc:503] Master 127.3.139.126:43361 was elected leader, sending a full tablet report...
I20240502 07:59:33.459100 19511 ts_tablet_manager.cc:902] T 0a10885caf66428880d2c862ca97f8db P 6fd33d34de9c463183177f1a13d4f851: Initiating tablet copy from peer d52c06abbcb44881a4837273c1ec8e91 (127.3.139.65:40139)
I20240502 07:59:33.459779 19511 tablet_copy_client.cc:308] T 0a10885caf66428880d2c862ca97f8db P 6fd33d34de9c463183177f1a13d4f851: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.65:40139
I20240502 07:59:33.462987 18417 tablet_copy_service.cc:140] P d52c06abbcb44881a4837273c1ec8e91: Received BeginTabletCopySession request for tablet 0a10885caf66428880d2c862ca97f8db from peer 6fd33d34de9c463183177f1a13d4f851 ({username='slave'} at 127.3.139.66:41273)
I20240502 07:59:33.463141 18417 tablet_copy_service.cc:161] P d52c06abbcb44881a4837273c1ec8e91: Beginning new tablet copy session on tablet 0a10885caf66428880d2c862ca97f8db from peer 6fd33d34de9c463183177f1a13d4f851 at {username='slave'} at 127.3.139.66:41273: session id = 6fd33d34de9c463183177f1a13d4f851-0a10885caf66428880d2c862ca97f8db
I20240502 07:59:33.464186 18417 tablet_copy_source_session.cc:205] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:33.467139 19511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a10885caf66428880d2c862ca97f8db. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:33.471808 19511 tablet_copy_client.cc:783] T 0a10885caf66428880d2c862ca97f8db P 6fd33d34de9c463183177f1a13d4f851: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:33.471971 19511 tablet_copy_client.cc:647] T 0a10885caf66428880d2c862ca97f8db P 6fd33d34de9c463183177f1a13d4f851: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:33.473379 19511 tablet_copy_client.cc:523] T 0a10885caf66428880d2c862ca97f8db P 6fd33d34de9c463183177f1a13d4f851: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:33.474675 19511 tablet_bootstrap.cc:492] T 0a10885caf66428880d2c862ca97f8db P 6fd33d34de9c463183177f1a13d4f851: Bootstrap starting.
I20240502 07:59:33.479478 19511 log.cc:824] T 0a10885caf66428880d2c862ca97f8db P 6fd33d34de9c463183177f1a13d4f851: Log is configured to *not* fsync() on all Append() calls
I20240502 07:59:33.496809 19511 tablet_bootstrap.cc:492] T 0a10885caf66428880d2c862ca97f8db P 6fd33d34de9c463183177f1a13d4f851: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:33.497051 19511 tablet_bootstrap.cc:492] T 0a10885caf66428880d2c862ca97f8db P 6fd33d34de9c463183177f1a13d4f851: Bootstrap complete.
I20240502 07:59:33.497272 19511 ts_tablet_manager.cc:1367] T 0a10885caf66428880d2c862ca97f8db P 6fd33d34de9c463183177f1a13d4f851: Time spent bootstrapping tablet: real 0.023s user 0.018s sys 0.003s
I20240502 07:59:33.498952 19511 raft_consensus.cc:348] T 0a10885caf66428880d2c862ca97f8db P 6fd33d34de9c463183177f1a13d4f851 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } attrs { replace: true } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: NON_VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: true } }
I20240502 07:59:33.499179 19511 raft_consensus.cc:729] T 0a10885caf66428880d2c862ca97f8db P 6fd33d34de9c463183177f1a13d4f851 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6fd33d34de9c463183177f1a13d4f851, State: Initialized, Role: LEARNER
I20240502 07:59:33.499449 19511 consensus_queue.cc:260] T 0a10885caf66428880d2c862ca97f8db P 6fd33d34de9c463183177f1a13d4f851 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } attrs { replace: true } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: NON_VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: true } }
I20240502 07:59:33.500275 19511 ts_tablet_manager.cc:1398] T 0a10885caf66428880d2c862ca97f8db P 6fd33d34de9c463183177f1a13d4f851: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20240502 07:59:33.501034 18417 tablet_copy_service.cc:342] P d52c06abbcb44881a4837273c1ec8e91: Request end of tablet copy session 6fd33d34de9c463183177f1a13d4f851-0a10885caf66428880d2c862ca97f8db received from {username='slave'} at 127.3.139.66:41273
I20240502 07:59:33.501107 18417 tablet_copy_service.cc:433] P d52c06abbcb44881a4837273c1ec8e91: ending tablet copy session 6fd33d34de9c463183177f1a13d4f851-0a10885caf66428880d2c862ca97f8db on tablet 0a10885caf66428880d2c862ca97f8db with peer 6fd33d34de9c463183177f1a13d4f851
I20240502 07:59:33.502432 19519 ts_tablet_manager.cc:902] T bd37e09a008c4e85804e041122ea1c57 P db667da4313e43e2917b5acc7e6f645d: Initiating tablet copy from peer d52c06abbcb44881a4837273c1ec8e91 (127.3.139.65:40139)
I20240502 07:59:33.503123 19519 tablet_copy_client.cc:308] T bd37e09a008c4e85804e041122ea1c57 P db667da4313e43e2917b5acc7e6f645d: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.65:40139
I20240502 07:59:33.504325 19525 ts_tablet_manager.cc:902] T 23442fad97424234ab659f3c0aecd0d0 P db667da4313e43e2917b5acc7e6f645d: Initiating tablet copy from peer d52c06abbcb44881a4837273c1ec8e91 (127.3.139.65:40139)
I20240502 07:59:33.504776 19525 tablet_copy_client.cc:308] T 23442fad97424234ab659f3c0aecd0d0 P db667da4313e43e2917b5acc7e6f645d: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.65:40139
I20240502 07:59:33.506464 18417 tablet_copy_service.cc:140] P d52c06abbcb44881a4837273c1ec8e91: Received BeginTabletCopySession request for tablet bd37e09a008c4e85804e041122ea1c57 from peer db667da4313e43e2917b5acc7e6f645d ({username='slave'} at 127.3.139.67:52915)
I20240502 07:59:33.506541 18417 tablet_copy_service.cc:161] P d52c06abbcb44881a4837273c1ec8e91: Beginning new tablet copy session on tablet bd37e09a008c4e85804e041122ea1c57 from peer db667da4313e43e2917b5acc7e6f645d at {username='slave'} at 127.3.139.67:52915: session id = db667da4313e43e2917b5acc7e6f645d-bd37e09a008c4e85804e041122ea1c57
I20240502 07:59:33.506561 18416 tablet_copy_service.cc:140] P d52c06abbcb44881a4837273c1ec8e91: Received BeginTabletCopySession request for tablet 23442fad97424234ab659f3c0aecd0d0 from peer db667da4313e43e2917b5acc7e6f645d ({username='slave'} at 127.3.139.67:52915)
I20240502 07:59:33.506654 18416 tablet_copy_service.cc:161] P d52c06abbcb44881a4837273c1ec8e91: Beginning new tablet copy session on tablet 23442fad97424234ab659f3c0aecd0d0 from peer db667da4313e43e2917b5acc7e6f645d at {username='slave'} at 127.3.139.67:52915: session id = db667da4313e43e2917b5acc7e6f645d-23442fad97424234ab659f3c0aecd0d0
I20240502 07:59:33.507227 18417 tablet_copy_source_session.cc:205] T bd37e09a008c4e85804e041122ea1c57 P d52c06abbcb44881a4837273c1ec8e91: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:33.507375 18416 tablet_copy_source_session.cc:205] T 23442fad97424234ab659f3c0aecd0d0 P d52c06abbcb44881a4837273c1ec8e91: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:33.509963 19525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23442fad97424234ab659f3c0aecd0d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:33.510248 19519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd37e09a008c4e85804e041122ea1c57. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:33.515937 19525 tablet_copy_client.cc:783] T 23442fad97424234ab659f3c0aecd0d0 P db667da4313e43e2917b5acc7e6f645d: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:33.515971 19519 tablet_copy_client.cc:783] T bd37e09a008c4e85804e041122ea1c57 P db667da4313e43e2917b5acc7e6f645d: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:33.516119 19519 tablet_copy_client.cc:647] T bd37e09a008c4e85804e041122ea1c57 P db667da4313e43e2917b5acc7e6f645d: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:33.516144 19525 tablet_copy_client.cc:647] T 23442fad97424234ab659f3c0aecd0d0 P db667da4313e43e2917b5acc7e6f645d: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:33.517452 19519 tablet_copy_client.cc:523] T bd37e09a008c4e85804e041122ea1c57 P db667da4313e43e2917b5acc7e6f645d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:33.517449 19525 tablet_copy_client.cc:523] T 23442fad97424234ab659f3c0aecd0d0 P db667da4313e43e2917b5acc7e6f645d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:33.518893 19525 tablet_bootstrap.cc:492] T 23442fad97424234ab659f3c0aecd0d0 P db667da4313e43e2917b5acc7e6f645d: Bootstrap starting.
I20240502 07:59:33.518893 19519 tablet_bootstrap.cc:492] T bd37e09a008c4e85804e041122ea1c57 P db667da4313e43e2917b5acc7e6f645d: Bootstrap starting.
I20240502 07:59:33.523736 19525 log.cc:824] T 23442fad97424234ab659f3c0aecd0d0 P db667da4313e43e2917b5acc7e6f645d: Log is configured to *not* fsync() on all Append() calls
I20240502 07:59:33.523891 19511 ts_tablet_manager.cc:902] T d1ee0467bf17461c9692fa887cf79cc2 P 6fd33d34de9c463183177f1a13d4f851: Initiating tablet copy from peer dbe891b2f04e454db1c2b5de528e01c7 (127.3.139.68:43983)
I20240502 07:59:33.524320 19511 tablet_copy_client.cc:308] T d1ee0467bf17461c9692fa887cf79cc2 P 6fd33d34de9c463183177f1a13d4f851: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.68:43983
I20240502 07:59:33.528853 18813 tablet_copy_service.cc:140] P dbe891b2f04e454db1c2b5de528e01c7: Received BeginTabletCopySession request for tablet d1ee0467bf17461c9692fa887cf79cc2 from peer 6fd33d34de9c463183177f1a13d4f851 ({username='slave'} at 127.3.139.66:40437)
I20240502 07:59:33.528992 18813 tablet_copy_service.cc:161] P dbe891b2f04e454db1c2b5de528e01c7: Beginning new tablet copy session on tablet d1ee0467bf17461c9692fa887cf79cc2 from peer 6fd33d34de9c463183177f1a13d4f851 at {username='slave'} at 127.3.139.66:40437: session id = 6fd33d34de9c463183177f1a13d4f851-d1ee0467bf17461c9692fa887cf79cc2
I20240502 07:59:33.530277 18813 tablet_copy_source_session.cc:205] T d1ee0467bf17461c9692fa887cf79cc2 P dbe891b2f04e454db1c2b5de528e01c7: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:33.531306 19511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1ee0467bf17461c9692fa887cf79cc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:33.533262 19511 tablet_copy_client.cc:783] T d1ee0467bf17461c9692fa887cf79cc2 P 6fd33d34de9c463183177f1a13d4f851: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:33.533392 19511 tablet_copy_client.cc:647] T d1ee0467bf17461c9692fa887cf79cc2 P 6fd33d34de9c463183177f1a13d4f851: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:33.534727 19511 tablet_copy_client.cc:523] T d1ee0467bf17461c9692fa887cf79cc2 P 6fd33d34de9c463183177f1a13d4f851: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:33.535847 19511 tablet_bootstrap.cc:492] T d1ee0467bf17461c9692fa887cf79cc2 P 6fd33d34de9c463183177f1a13d4f851: Bootstrap starting.
I20240502 07:59:33.536679 19266 heartbeater.cc:503] Master 127.3.139.126:43361 was elected leader, sending a full tablet report...
I20240502 07:59:33.547120 19525 tablet_bootstrap.cc:492] T 23442fad97424234ab659f3c0aecd0d0 P db667da4313e43e2917b5acc7e6f645d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:33.547423 19525 tablet_bootstrap.cc:492] T 23442fad97424234ab659f3c0aecd0d0 P db667da4313e43e2917b5acc7e6f645d: Bootstrap complete.
I20240502 07:59:33.547685 19525 ts_tablet_manager.cc:1367] T 23442fad97424234ab659f3c0aecd0d0 P db667da4313e43e2917b5acc7e6f645d: Time spent bootstrapping tablet: real 0.029s user 0.027s sys 0.000s
I20240502 07:59:33.549973 19525 raft_consensus.cc:348] T 23442fad97424234ab659f3c0aecd0d0 P db667da4313e43e2917b5acc7e6f645d [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } attrs { replace: true } } peers { permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: NON_VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: true } }
I20240502 07:59:33.550201 19525 raft_consensus.cc:729] T 23442fad97424234ab659f3c0aecd0d0 P db667da4313e43e2917b5acc7e6f645d [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: db667da4313e43e2917b5acc7e6f645d, State: Initialized, Role: LEARNER
I20240502 07:59:33.550539 19525 consensus_queue.cc:260] T 23442fad97424234ab659f3c0aecd0d0 P db667da4313e43e2917b5acc7e6f645d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } attrs { replace: true } } peers { permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: NON_VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: true } }
I20240502 07:59:33.550598 19519 tablet_bootstrap.cc:492] T bd37e09a008c4e85804e041122ea1c57 P db667da4313e43e2917b5acc7e6f645d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:33.550860 19519 tablet_bootstrap.cc:492] T bd37e09a008c4e85804e041122ea1c57 P db667da4313e43e2917b5acc7e6f645d: Bootstrap complete.
I20240502 07:59:33.550993 19519 ts_tablet_manager.cc:1367] T bd37e09a008c4e85804e041122ea1c57 P db667da4313e43e2917b5acc7e6f645d: Time spent bootstrapping tablet: real 0.032s user 0.019s sys 0.004s
I20240502 07:59:33.551323 19525 ts_tablet_manager.cc:1398] T 23442fad97424234ab659f3c0aecd0d0 P db667da4313e43e2917b5acc7e6f645d: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20240502 07:59:33.551450 19519 raft_consensus.cc:348] T bd37e09a008c4e85804e041122ea1c57 P db667da4313e43e2917b5acc7e6f645d [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } attrs { replace: true } } peers { permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: NON_VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: true } }
I20240502 07:59:33.551554 19519 raft_consensus.cc:729] T bd37e09a008c4e85804e041122ea1c57 P db667da4313e43e2917b5acc7e6f645d [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: db667da4313e43e2917b5acc7e6f645d, State: Initialized, Role: LEARNER
I20240502 07:59:33.551654 19519 consensus_queue.cc:260] T bd37e09a008c4e85804e041122ea1c57 P db667da4313e43e2917b5acc7e6f645d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } attrs { replace: true } } peers { permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: NON_VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: true } }
I20240502 07:59:33.551954 19519 ts_tablet_manager.cc:1398] T bd37e09a008c4e85804e041122ea1c57 P db667da4313e43e2917b5acc7e6f645d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:33.552412 18416 tablet_copy_service.cc:342] P d52c06abbcb44881a4837273c1ec8e91: Request end of tablet copy session db667da4313e43e2917b5acc7e6f645d-bd37e09a008c4e85804e041122ea1c57 received from {username='slave'} at 127.3.139.67:52915
I20240502 07:59:33.552481 18416 tablet_copy_service.cc:433] P d52c06abbcb44881a4837273c1ec8e91: ending tablet copy session db667da4313e43e2917b5acc7e6f645d-bd37e09a008c4e85804e041122ea1c57 on tablet bd37e09a008c4e85804e041122ea1c57 with peer db667da4313e43e2917b5acc7e6f645d
I20240502 07:59:33.554067 18416 tablet_copy_service.cc:342] P d52c06abbcb44881a4837273c1ec8e91: Request end of tablet copy session db667da4313e43e2917b5acc7e6f645d-23442fad97424234ab659f3c0aecd0d0 received from {username='slave'} at 127.3.139.67:52915
I20240502 07:59:33.554142 18416 tablet_copy_service.cc:433] P d52c06abbcb44881a4837273c1ec8e91: ending tablet copy session db667da4313e43e2917b5acc7e6f645d-23442fad97424234ab659f3c0aecd0d0 on tablet 23442fad97424234ab659f3c0aecd0d0 with peer db667da4313e43e2917b5acc7e6f645d
I20240502 07:59:33.562081 19525 ts_tablet_manager.cc:902] T 58aa0644416e4e698766581b577d21d0 P db667da4313e43e2917b5acc7e6f645d: Initiating tablet copy from peer d52c06abbcb44881a4837273c1ec8e91 (127.3.139.65:40139)
I20240502 07:59:33.562435 19525 tablet_copy_client.cc:308] T 58aa0644416e4e698766581b577d21d0 P db667da4313e43e2917b5acc7e6f645d: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.65:40139
I20240502 07:59:33.564224 18416 tablet_copy_service.cc:140] P d52c06abbcb44881a4837273c1ec8e91: Received BeginTabletCopySession request for tablet 58aa0644416e4e698766581b577d21d0 from peer db667da4313e43e2917b5acc7e6f645d ({username='slave'} at 127.3.139.67:52915)
I20240502 07:59:33.564319 18416 tablet_copy_service.cc:161] P d52c06abbcb44881a4837273c1ec8e91: Beginning new tablet copy session on tablet 58aa0644416e4e698766581b577d21d0 from peer db667da4313e43e2917b5acc7e6f645d at {username='slave'} at 127.3.139.67:52915: session id = db667da4313e43e2917b5acc7e6f645d-58aa0644416e4e698766581b577d21d0
I20240502 07:59:33.565160 18416 tablet_copy_source_session.cc:205] T 58aa0644416e4e698766581b577d21d0 P d52c06abbcb44881a4837273c1ec8e91: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:33.565871 19525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58aa0644416e4e698766581b577d21d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:33.566943 19511 tablet_bootstrap.cc:492] T d1ee0467bf17461c9692fa887cf79cc2 P 6fd33d34de9c463183177f1a13d4f851: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:33.567227 19511 tablet_bootstrap.cc:492] T d1ee0467bf17461c9692fa887cf79cc2 P 6fd33d34de9c463183177f1a13d4f851: Bootstrap complete.
I20240502 07:59:33.567384 19525 tablet_copy_client.cc:783] T 58aa0644416e4e698766581b577d21d0 P db667da4313e43e2917b5acc7e6f645d: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:33.567389 19511 ts_tablet_manager.cc:1367] T d1ee0467bf17461c9692fa887cf79cc2 P 6fd33d34de9c463183177f1a13d4f851: Time spent bootstrapping tablet: real 0.032s user 0.020s sys 0.003s
I20240502 07:59:33.567508 19525 tablet_copy_client.cc:647] T 58aa0644416e4e698766581b577d21d0 P db667da4313e43e2917b5acc7e6f645d: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:33.567901 19511 raft_consensus.cc:348] T d1ee0467bf17461c9692fa887cf79cc2 P 6fd33d34de9c463183177f1a13d4f851 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } attrs { replace: true } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: NON_VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: true } }
I20240502 07:59:33.568011 19511 raft_consensus.cc:729] T d1ee0467bf17461c9692fa887cf79cc2 P 6fd33d34de9c463183177f1a13d4f851 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6fd33d34de9c463183177f1a13d4f851, State: Initialized, Role: LEARNER
I20240502 07:59:33.568105 19511 consensus_queue.cc:260] T d1ee0467bf17461c9692fa887cf79cc2 P 6fd33d34de9c463183177f1a13d4f851 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } attrs { replace: true } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: NON_VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: true } }
I20240502 07:59:33.568380 19511 ts_tablet_manager.cc:1398] T d1ee0467bf17461c9692fa887cf79cc2 P 6fd33d34de9c463183177f1a13d4f851: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:33.568636 19525 tablet_copy_client.cc:523] T 58aa0644416e4e698766581b577d21d0 P db667da4313e43e2917b5acc7e6f645d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:33.568881 18813 tablet_copy_service.cc:342] P dbe891b2f04e454db1c2b5de528e01c7: Request end of tablet copy session 6fd33d34de9c463183177f1a13d4f851-d1ee0467bf17461c9692fa887cf79cc2 received from {username='slave'} at 127.3.139.66:40437
I20240502 07:59:33.568964 18813 tablet_copy_service.cc:433] P dbe891b2f04e454db1c2b5de528e01c7: ending tablet copy session 6fd33d34de9c463183177f1a13d4f851-d1ee0467bf17461c9692fa887cf79cc2 on tablet d1ee0467bf17461c9692fa887cf79cc2 with peer 6fd33d34de9c463183177f1a13d4f851
I20240502 07:59:33.569675 19525 tablet_bootstrap.cc:492] T 58aa0644416e4e698766581b577d21d0 P db667da4313e43e2917b5acc7e6f645d: Bootstrap starting.
I20240502 07:59:33.586892 19525 tablet_bootstrap.cc:492] T 58aa0644416e4e698766581b577d21d0 P db667da4313e43e2917b5acc7e6f645d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:33.587164 19525 tablet_bootstrap.cc:492] T 58aa0644416e4e698766581b577d21d0 P db667da4313e43e2917b5acc7e6f645d: Bootstrap complete.
I20240502 07:59:33.587316 19525 ts_tablet_manager.cc:1367] T 58aa0644416e4e698766581b577d21d0 P db667da4313e43e2917b5acc7e6f645d: Time spent bootstrapping tablet: real 0.018s user 0.013s sys 0.005s
I20240502 07:59:33.587656 19525 raft_consensus.cc:348] T 58aa0644416e4e698766581b577d21d0 P db667da4313e43e2917b5acc7e6f645d [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } attrs { replace: true } } peers { permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: NON_VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: true } }
I20240502 07:59:33.587746 19525 raft_consensus.cc:729] T 58aa0644416e4e698766581b577d21d0 P db667da4313e43e2917b5acc7e6f645d [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: db667da4313e43e2917b5acc7e6f645d, State: Initialized, Role: LEARNER
I20240502 07:59:33.587833 19525 consensus_queue.cc:260] T 58aa0644416e4e698766581b577d21d0 P db667da4313e43e2917b5acc7e6f645d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } attrs { replace: true } } peers { permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: NON_VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: true } }
I20240502 07:59:33.588042 19525 ts_tablet_manager.cc:1398] T 58aa0644416e4e698766581b577d21d0 P db667da4313e43e2917b5acc7e6f645d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:33.588444 18416 tablet_copy_service.cc:342] P d52c06abbcb44881a4837273c1ec8e91: Request end of tablet copy session db667da4313e43e2917b5acc7e6f645d-58aa0644416e4e698766581b577d21d0 received from {username='slave'} at 127.3.139.67:52915
I20240502 07:59:33.588523 18416 tablet_copy_service.cc:433] P d52c06abbcb44881a4837273c1ec8e91: ending tablet copy session db667da4313e43e2917b5acc7e6f645d-58aa0644416e4e698766581b577d21d0 on tablet 58aa0644416e4e698766581b577d21d0 with peer db667da4313e43e2917b5acc7e6f645d
I20240502 07:59:33.591662 19511 ts_tablet_manager.cc:902] T d8c026e315ac4622b77b618de5c0b09f P 6fd33d34de9c463183177f1a13d4f851: Initiating tablet copy from peer dbe891b2f04e454db1c2b5de528e01c7 (127.3.139.68:43983)
I20240502 07:59:33.592037 19511 tablet_copy_client.cc:308] T d8c026e315ac4622b77b618de5c0b09f P 6fd33d34de9c463183177f1a13d4f851: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.68:43983
I20240502 07:59:33.592430 18813 tablet_copy_service.cc:140] P dbe891b2f04e454db1c2b5de528e01c7: Received BeginTabletCopySession request for tablet d8c026e315ac4622b77b618de5c0b09f from peer 6fd33d34de9c463183177f1a13d4f851 ({username='slave'} at 127.3.139.66:40437)
I20240502 07:59:33.592516 18813 tablet_copy_service.cc:161] P dbe891b2f04e454db1c2b5de528e01c7: Beginning new tablet copy session on tablet d8c026e315ac4622b77b618de5c0b09f from peer 6fd33d34de9c463183177f1a13d4f851 at {username='slave'} at 127.3.139.66:40437: session id = 6fd33d34de9c463183177f1a13d4f851-d8c026e315ac4622b77b618de5c0b09f
I20240502 07:59:33.593163 18813 tablet_copy_source_session.cc:205] T d8c026e315ac4622b77b618de5c0b09f P dbe891b2f04e454db1c2b5de528e01c7: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:33.593842 19511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8c026e315ac4622b77b618de5c0b09f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:33.595479 19511 tablet_copy_client.cc:783] T d8c026e315ac4622b77b618de5c0b09f P 6fd33d34de9c463183177f1a13d4f851: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:33.595618 19511 tablet_copy_client.cc:647] T d8c026e315ac4622b77b618de5c0b09f P 6fd33d34de9c463183177f1a13d4f851: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:33.596614 19511 tablet_copy_client.cc:523] T d8c026e315ac4622b77b618de5c0b09f P 6fd33d34de9c463183177f1a13d4f851: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:33.597512 19511 tablet_bootstrap.cc:492] T d8c026e315ac4622b77b618de5c0b09f P 6fd33d34de9c463183177f1a13d4f851: Bootstrap starting.
I20240502 07:59:33.614370 19511 tablet_bootstrap.cc:492] T d8c026e315ac4622b77b618de5c0b09f P 6fd33d34de9c463183177f1a13d4f851: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:33.614550 19511 tablet_bootstrap.cc:492] T d8c026e315ac4622b77b618de5c0b09f P 6fd33d34de9c463183177f1a13d4f851: Bootstrap complete.
I20240502 07:59:33.614665 19511 ts_tablet_manager.cc:1367] T d8c026e315ac4622b77b618de5c0b09f P 6fd33d34de9c463183177f1a13d4f851: Time spent bootstrapping tablet: real 0.017s user 0.014s sys 0.000s
I20240502 07:59:33.614980 19511 raft_consensus.cc:348] T d8c026e315ac4622b77b618de5c0b09f P 6fd33d34de9c463183177f1a13d4f851 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } attrs { replace: true } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: NON_VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: true } }
I20240502 07:59:33.615092 19511 raft_consensus.cc:729] T d8c026e315ac4622b77b618de5c0b09f P 6fd33d34de9c463183177f1a13d4f851 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6fd33d34de9c463183177f1a13d4f851, State: Initialized, Role: LEARNER
I20240502 07:59:33.615178 19511 consensus_queue.cc:260] T d8c026e315ac4622b77b618de5c0b09f P 6fd33d34de9c463183177f1a13d4f851 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } attrs { replace: true } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: NON_VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: true } }
I20240502 07:59:33.615408 19511 ts_tablet_manager.cc:1398] T d8c026e315ac4622b77b618de5c0b09f P 6fd33d34de9c463183177f1a13d4f851: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240502 07:59:33.615814 18813 tablet_copy_service.cc:342] P dbe891b2f04e454db1c2b5de528e01c7: Request end of tablet copy session 6fd33d34de9c463183177f1a13d4f851-d8c026e315ac4622b77b618de5c0b09f received from {username='slave'} at 127.3.139.66:40437
I20240502 07:59:33.615878 18813 tablet_copy_service.cc:433] P dbe891b2f04e454db1c2b5de528e01c7: ending tablet copy session 6fd33d34de9c463183177f1a13d4f851-d8c026e315ac4622b77b618de5c0b09f on tablet d8c026e315ac4622b77b618de5c0b09f with peer 6fd33d34de9c463183177f1a13d4f851
I20240502 07:59:33.895315 19220 raft_consensus.cc:1203] T bd37e09a008c4e85804e041122ea1c57 P db667da4313e43e2917b5acc7e6f645d [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:33.916419 19089 raft_consensus.cc:1203] T 0a10885caf66428880d2c862ca97f8db P 6fd33d34de9c463183177f1a13d4f851 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:33.936050 19220 raft_consensus.cc:1203] T 23442fad97424234ab659f3c0aecd0d0 P db667da4313e43e2917b5acc7e6f645d [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:33.973052 19220 raft_consensus.cc:1203] T 58aa0644416e4e698766581b577d21d0 P db667da4313e43e2917b5acc7e6f645d [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:34.061821 19089 raft_consensus.cc:1203] T d8c026e315ac4622b77b618de5c0b09f P 6fd33d34de9c463183177f1a13d4f851 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:34.125649 19089 raft_consensus.cc:1203] T d1ee0467bf17461c9692fa887cf79cc2 P 6fd33d34de9c463183177f1a13d4f851 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:34.388737 19610 raft_consensus.cc:1050] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91: attempting to promote NON_VOTER 6fd33d34de9c463183177f1a13d4f851 to VOTER
I20240502 07:59:34.389115 19610 consensus_queue.cc:237] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } attrs { replace: true } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: false } }
I20240502 07:59:34.390250 18790 raft_consensus.cc:1261] T 0a10885caf66428880d2c862ca97f8db P dbe891b2f04e454db1c2b5de528e01c7 [term 1 FOLLOWER]: Refusing update from remote peer d52c06abbcb44881a4837273c1ec8e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:34.390282 19089 raft_consensus.cc:1261] T 0a10885caf66428880d2c862ca97f8db P 6fd33d34de9c463183177f1a13d4f851 [term 1 LEARNER]: Refusing update from remote peer d52c06abbcb44881a4837273c1ec8e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:34.390661 19409 consensus_queue.cc:1035] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:34.390838 19609 consensus_queue.cc:1035] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:34.392426 19609 raft_consensus.cc:2932] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 6fd33d34de9c463183177f1a13d4f851 (127.3.139.66) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } attrs { replace: true } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: false } } }
I20240502 07:59:34.392828 18790 raft_consensus.cc:2932] T 0a10885caf66428880d2c862ca97f8db P dbe891b2f04e454db1c2b5de528e01c7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 6fd33d34de9c463183177f1a13d4f851 (127.3.139.66) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } attrs { replace: true } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: false } } }
I20240502 07:59:34.394658 18249 catalog_manager.cc:5484] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91 reported cstate change: config changed from index 4 to 5, 6fd33d34de9c463183177f1a13d4f851 (127.3.139.66) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d52c06abbcb44881a4837273c1ec8e91" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:34.395548 19089 raft_consensus.cc:2932] T 0a10885caf66428880d2c862ca97f8db P 6fd33d34de9c463183177f1a13d4f851 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 6fd33d34de9c463183177f1a13d4f851 (127.3.139.66) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } attrs { replace: true } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: false } } }
I20240502 07:59:34.434937 19609 raft_consensus.cc:1050] T bd37e09a008c4e85804e041122ea1c57 P d52c06abbcb44881a4837273c1ec8e91: attempting to promote NON_VOTER db667da4313e43e2917b5acc7e6f645d to VOTER
I20240502 07:59:34.435263 19609 consensus_queue.cc:237] T bd37e09a008c4e85804e041122ea1c57 P d52c06abbcb44881a4837273c1ec8e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } attrs { replace: true } } peers { permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: false } }
I20240502 07:59:34.436353 18790 raft_consensus.cc:1261] T bd37e09a008c4e85804e041122ea1c57 P dbe891b2f04e454db1c2b5de528e01c7 [term 1 FOLLOWER]: Refusing update from remote peer d52c06abbcb44881a4837273c1ec8e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:34.436390 19220 raft_consensus.cc:1261] T bd37e09a008c4e85804e041122ea1c57 P db667da4313e43e2917b5acc7e6f645d [term 1 LEARNER]: Refusing update from remote peer d52c06abbcb44881a4837273c1ec8e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:34.436723 19676 consensus_queue.cc:1035] T bd37e09a008c4e85804e041122ea1c57 P d52c06abbcb44881a4837273c1ec8e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:34.436928 19609 consensus_queue.cc:1035] T bd37e09a008c4e85804e041122ea1c57 P d52c06abbcb44881a4837273c1ec8e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:34.438625 19609 raft_consensus.cc:2932] T bd37e09a008c4e85804e041122ea1c57 P d52c06abbcb44881a4837273c1ec8e91 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, db667da4313e43e2917b5acc7e6f645d (127.3.139.67) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } attrs { replace: true } } peers { permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: false } } }
I20240502 07:59:34.439066 18790 raft_consensus.cc:2932] T bd37e09a008c4e85804e041122ea1c57 P dbe891b2f04e454db1c2b5de528e01c7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, db667da4313e43e2917b5acc7e6f645d (127.3.139.67) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } attrs { replace: true } } peers { permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: false } } }
I20240502 07:59:34.440857 18250 catalog_manager.cc:5484] T bd37e09a008c4e85804e041122ea1c57 P d52c06abbcb44881a4837273c1ec8e91 reported cstate change: config changed from index 4 to 5, db667da4313e43e2917b5acc7e6f645d (127.3.139.67) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d52c06abbcb44881a4837273c1ec8e91" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:34.440901 19220 raft_consensus.cc:2932] T bd37e09a008c4e85804e041122ea1c57 P db667da4313e43e2917b5acc7e6f645d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, db667da4313e43e2917b5acc7e6f645d (127.3.139.67) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } attrs { replace: true } } peers { permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: false } } }
I20240502 07:59:34.444820 18396 consensus_queue.cc:237] T bd37e09a008c4e85804e041122ea1c57 P d52c06abbcb44881a4837273c1ec8e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: false } }
I20240502 07:59:34.445611 19220 raft_consensus.cc:1261] T bd37e09a008c4e85804e041122ea1c57 P db667da4313e43e2917b5acc7e6f645d [term 1 FOLLOWER]: Refusing update from remote peer d52c06abbcb44881a4837273c1ec8e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:34.445916 19409 consensus_queue.cc:1035] T bd37e09a008c4e85804e041122ea1c57 P d52c06abbcb44881a4837273c1ec8e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:34.447175 19609 raft_consensus.cc:2932] T bd37e09a008c4e85804e041122ea1c57 P d52c06abbcb44881a4837273c1ec8e91 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER dbe891b2f04e454db1c2b5de528e01c7 (127.3.139.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: false } } }
I20240502 07:59:34.447577 19220 raft_consensus.cc:2932] T bd37e09a008c4e85804e041122ea1c57 P db667da4313e43e2917b5acc7e6f645d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER dbe891b2f04e454db1c2b5de528e01c7 (127.3.139.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: false } } }
I20240502 07:59:34.448199 18242 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet bd37e09a008c4e85804e041122ea1c57 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:34.448952 18251 catalog_manager.cc:5484] T bd37e09a008c4e85804e041122ea1c57 P d52c06abbcb44881a4837273c1ec8e91 reported cstate change: config changed from index 5 to 6, VOTER dbe891b2f04e454db1c2b5de528e01c7 (127.3.139.68) evicted. New cstate: current_term: 1 leader_uuid: "d52c06abbcb44881a4837273c1ec8e91" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:34.450888 18771 tablet_service.cc:1514] Processing DeleteTablet for tablet bd37e09a008c4e85804e041122ea1c57 with delete_type TABLET_DATA_TOMBSTONED (TS dbe891b2f04e454db1c2b5de528e01c7 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:37726
I20240502 07:59:34.451452 19684 tablet_replica.cc:330] T bd37e09a008c4e85804e041122ea1c57 P dbe891b2f04e454db1c2b5de528e01c7: stopping tablet replica
I20240502 07:59:34.451699 19684 raft_consensus.cc:2229] T bd37e09a008c4e85804e041122ea1c57 P dbe891b2f04e454db1c2b5de528e01c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:34.451884 19684 raft_consensus.cc:2258] T bd37e09a008c4e85804e041122ea1c57 P dbe891b2f04e454db1c2b5de528e01c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:34.452762 19684 ts_tablet_manager.cc:1875] T bd37e09a008c4e85804e041122ea1c57 P dbe891b2f04e454db1c2b5de528e01c7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:34.454334 19684 ts_tablet_manager.cc:1888] T bd37e09a008c4e85804e041122ea1c57 P dbe891b2f04e454db1c2b5de528e01c7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 07:59:34.454411 19684 log.cc:1196] T bd37e09a008c4e85804e041122ea1c57 P dbe891b2f04e454db1c2b5de528e01c7: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal/wals/bd37e09a008c4e85804e041122ea1c57
I20240502 07:59:34.454898 18241 catalog_manager.cc:4830] TS dbe891b2f04e454db1c2b5de528e01c7 (127.3.139.68:43983): tablet bd37e09a008c4e85804e041122ea1c57 (table rebalance_test_table_1 [id=30b6f8cef2c1488fadb7fd8c388373c5]) successfully deleted
I20240502 07:59:34.488221 18396 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0a10885caf66428880d2c862ca97f8db"
dest_uuid: "d52c06abbcb44881a4837273c1ec8e91"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:44438
I20240502 07:59:34.488355 18396 raft_consensus.cc:595] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91 [term 1 LEADER]: Received request to transfer leadership
I20240502 07:59:34.495224 19409 raft_consensus.cc:1050] T 58aa0644416e4e698766581b577d21d0 P d52c06abbcb44881a4837273c1ec8e91: attempting to promote NON_VOTER db667da4313e43e2917b5acc7e6f645d to VOTER
I20240502 07:59:34.495563 19409 consensus_queue.cc:237] T 58aa0644416e4e698766581b577d21d0 P d52c06abbcb44881a4837273c1ec8e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } attrs { replace: true } } peers { permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: false } }
I20240502 07:59:34.496634 18790 raft_consensus.cc:1261] T 58aa0644416e4e698766581b577d21d0 P dbe891b2f04e454db1c2b5de528e01c7 [term 1 FOLLOWER]: Refusing update from remote peer d52c06abbcb44881a4837273c1ec8e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:34.496644 19220 raft_consensus.cc:1261] T 58aa0644416e4e698766581b577d21d0 P db667da4313e43e2917b5acc7e6f645d [term 1 LEARNER]: Refusing update from remote peer d52c06abbcb44881a4837273c1ec8e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:34.497063 19676 consensus_queue.cc:1035] T 58aa0644416e4e698766581b577d21d0 P d52c06abbcb44881a4837273c1ec8e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:34.497236 19409 consensus_queue.cc:1035] T 58aa0644416e4e698766581b577d21d0 P d52c06abbcb44881a4837273c1ec8e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:34.499102 19676 raft_consensus.cc:2932] T 58aa0644416e4e698766581b577d21d0 P d52c06abbcb44881a4837273c1ec8e91 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, db667da4313e43e2917b5acc7e6f645d (127.3.139.67) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } attrs { replace: true } } peers { permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: false } } }
I20240502 07:59:34.499452 18790 raft_consensus.cc:2932] T 58aa0644416e4e698766581b577d21d0 P dbe891b2f04e454db1c2b5de528e01c7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, db667da4313e43e2917b5acc7e6f645d (127.3.139.67) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } attrs { replace: true } } peers { permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: false } } }
I20240502 07:59:34.501470 19220 raft_consensus.cc:2932] T 58aa0644416e4e698766581b577d21d0 P db667da4313e43e2917b5acc7e6f645d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, db667da4313e43e2917b5acc7e6f645d (127.3.139.67) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } attrs { replace: true } } peers { permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: false } } }
I20240502 07:59:34.501500 18249 catalog_manager.cc:5484] T 58aa0644416e4e698766581b577d21d0 P d52c06abbcb44881a4837273c1ec8e91 reported cstate change: config changed from index 4 to 5, db667da4313e43e2917b5acc7e6f645d (127.3.139.67) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d52c06abbcb44881a4837273c1ec8e91" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:34.505837 18396 consensus_queue.cc:237] T 58aa0644416e4e698766581b577d21d0 P d52c06abbcb44881a4837273c1ec8e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: false } }
I20240502 07:59:34.506664 19220 raft_consensus.cc:1261] T 58aa0644416e4e698766581b577d21d0 P db667da4313e43e2917b5acc7e6f645d [term 1 FOLLOWER]: Refusing update from remote peer d52c06abbcb44881a4837273c1ec8e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:34.507014 19409 consensus_queue.cc:1035] T 58aa0644416e4e698766581b577d21d0 P d52c06abbcb44881a4837273c1ec8e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:34.508379 19609 raft_consensus.cc:2932] T 58aa0644416e4e698766581b577d21d0 P d52c06abbcb44881a4837273c1ec8e91 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER dbe891b2f04e454db1c2b5de528e01c7 (127.3.139.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: false } } }
I20240502 07:59:34.508733 19220 raft_consensus.cc:2932] T 58aa0644416e4e698766581b577d21d0 P db667da4313e43e2917b5acc7e6f645d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER dbe891b2f04e454db1c2b5de528e01c7 (127.3.139.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: false } } }
I20240502 07:59:34.510069 18242 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 58aa0644416e4e698766581b577d21d0 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:34.510639 18250 catalog_manager.cc:5484] T 58aa0644416e4e698766581b577d21d0 P db667da4313e43e2917b5acc7e6f645d reported cstate change: config changed from index 5 to 6, VOTER dbe891b2f04e454db1c2b5de528e01c7 (127.3.139.68) evicted. New cstate: current_term: 1 leader_uuid: "d52c06abbcb44881a4837273c1ec8e91" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: false } } }
I20240502 07:59:34.513137 18771 tablet_service.cc:1514] Processing DeleteTablet for tablet 58aa0644416e4e698766581b577d21d0 with delete_type TABLET_DATA_TOMBSTONED (TS dbe891b2f04e454db1c2b5de528e01c7 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:37726
I20240502 07:59:34.513324 19684 tablet_replica.cc:330] T 58aa0644416e4e698766581b577d21d0 P dbe891b2f04e454db1c2b5de528e01c7: stopping tablet replica
I20240502 07:59:34.513458 19684 raft_consensus.cc:2229] T 58aa0644416e4e698766581b577d21d0 P dbe891b2f04e454db1c2b5de528e01c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:34.513604 19684 raft_consensus.cc:2258] T 58aa0644416e4e698766581b577d21d0 P dbe891b2f04e454db1c2b5de528e01c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:34.514195 19684 ts_tablet_manager.cc:1875] T 58aa0644416e4e698766581b577d21d0 P dbe891b2f04e454db1c2b5de528e01c7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:34.515795 19684 ts_tablet_manager.cc:1888] T 58aa0644416e4e698766581b577d21d0 P dbe891b2f04e454db1c2b5de528e01c7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 07:59:34.515858 19684 log.cc:1196] T 58aa0644416e4e698766581b577d21d0 P dbe891b2f04e454db1c2b5de528e01c7: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal/wals/58aa0644416e4e698766581b577d21d0
I20240502 07:59:34.516240 18241 catalog_manager.cc:4830] TS dbe891b2f04e454db1c2b5de528e01c7 (127.3.139.68:43983): tablet 58aa0644416e4e698766581b577d21d0 (table rebalance_test_table_0 [id=d0fba7f1dfcc455d96585f048abd07a2]) successfully deleted
I20240502 07:59:34.520220 18397 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0a10885caf66428880d2c862ca97f8db"
dest_uuid: "d52c06abbcb44881a4837273c1ec8e91"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:44482
I20240502 07:59:34.520318 18397 raft_consensus.cc:595] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91 [term 1 LEADER]: Received request to transfer leadership
I20240502 07:59:34.524045 19610 raft_consensus.cc:1050] T 23442fad97424234ab659f3c0aecd0d0 P d52c06abbcb44881a4837273c1ec8e91: attempting to promote NON_VOTER db667da4313e43e2917b5acc7e6f645d to VOTER
I20240502 07:59:34.524467 19610 consensus_queue.cc:237] T 23442fad97424234ab659f3c0aecd0d0 P d52c06abbcb44881a4837273c1ec8e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } attrs { replace: true } } peers { permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: false } }
I20240502 07:59:34.525857 19220 raft_consensus.cc:1261] T 23442fad97424234ab659f3c0aecd0d0 P db667da4313e43e2917b5acc7e6f645d [term 1 LEARNER]: Refusing update from remote peer d52c06abbcb44881a4837273c1ec8e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:34.526126 18790 raft_consensus.cc:1261] T 23442fad97424234ab659f3c0aecd0d0 P dbe891b2f04e454db1c2b5de528e01c7 [term 1 FOLLOWER]: Refusing update from remote peer d52c06abbcb44881a4837273c1ec8e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:34.526154 19409 consensus_queue.cc:1035] T 23442fad97424234ab659f3c0aecd0d0 P d52c06abbcb44881a4837273c1ec8e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:34.526440 19409 consensus_queue.cc:1035] T 23442fad97424234ab659f3c0aecd0d0 P d52c06abbcb44881a4837273c1ec8e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:34.528183 19409 raft_consensus.cc:2932] T 23442fad97424234ab659f3c0aecd0d0 P d52c06abbcb44881a4837273c1ec8e91 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, db667da4313e43e2917b5acc7e6f645d (127.3.139.67) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } attrs { replace: true } } peers { permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: false } } }
I20240502 07:59:34.528589 19220 raft_consensus.cc:2932] T 23442fad97424234ab659f3c0aecd0d0 P db667da4313e43e2917b5acc7e6f645d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, db667da4313e43e2917b5acc7e6f645d (127.3.139.67) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } attrs { replace: true } } peers { permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: false } } }
I20240502 07:59:34.530468 18251 catalog_manager.cc:5484] T 23442fad97424234ab659f3c0aecd0d0 P d52c06abbcb44881a4837273c1ec8e91 reported cstate change: config changed from index 4 to 5, db667da4313e43e2917b5acc7e6f645d (127.3.139.67) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d52c06abbcb44881a4837273c1ec8e91" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:34.532822 18397 consensus_queue.cc:237] T 23442fad97424234ab659f3c0aecd0d0 P d52c06abbcb44881a4837273c1ec8e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: false } }
I20240502 07:59:34.533344 19642 raft_consensus.cc:1050] T d8c026e315ac4622b77b618de5c0b09f P dbe891b2f04e454db1c2b5de528e01c7: attempting to promote NON_VOTER 6fd33d34de9c463183177f1a13d4f851 to VOTER
I20240502 07:59:34.533689 19220 raft_consensus.cc:1261] T 23442fad97424234ab659f3c0aecd0d0 P db667da4313e43e2917b5acc7e6f645d [term 1 FOLLOWER]: Refusing update from remote peer d52c06abbcb44881a4837273c1ec8e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:34.533758 19642 consensus_queue.cc:237] T d8c026e315ac4622b77b618de5c0b09f P dbe891b2f04e454db1c2b5de528e01c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } attrs { replace: true } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: false } }
I20240502 07:59:34.533970 19610 consensus_queue.cc:1035] T 23442fad97424234ab659f3c0aecd0d0 P d52c06abbcb44881a4837273c1ec8e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:34.534868 18397 raft_consensus.cc:1261] T d8c026e315ac4622b77b618de5c0b09f P d52c06abbcb44881a4837273c1ec8e91 [term 1 FOLLOWER]: Refusing update from remote peer dbe891b2f04e454db1c2b5de528e01c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:34.535236 19089 raft_consensus.cc:1261] T d8c026e315ac4622b77b618de5c0b09f P 6fd33d34de9c463183177f1a13d4f851 [term 1 LEARNER]: Refusing update from remote peer dbe891b2f04e454db1c2b5de528e01c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:34.535292 19641 consensus_queue.cc:1035] T d8c026e315ac4622b77b618de5c0b09f P dbe891b2f04e454db1c2b5de528e01c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:34.535480 19610 raft_consensus.cc:2932] T 23442fad97424234ab659f3c0aecd0d0 P d52c06abbcb44881a4837273c1ec8e91 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER dbe891b2f04e454db1c2b5de528e01c7 (127.3.139.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: false } } }
I20240502 07:59:34.535799 19641 consensus_queue.cc:1035] T d8c026e315ac4622b77b618de5c0b09f P dbe891b2f04e454db1c2b5de528e01c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:34.536931 18242 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 23442fad97424234ab659f3c0aecd0d0 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:34.536938 19220 raft_consensus.cc:2932] T 23442fad97424234ab659f3c0aecd0d0 P db667da4313e43e2917b5acc7e6f645d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER dbe891b2f04e454db1c2b5de528e01c7 (127.3.139.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: false } } }
I20240502 07:59:34.537801 18250 catalog_manager.cc:5484] T 23442fad97424234ab659f3c0aecd0d0 P d52c06abbcb44881a4837273c1ec8e91 reported cstate change: config changed from index 5 to 6, VOTER dbe891b2f04e454db1c2b5de528e01c7 (127.3.139.68) evicted. New cstate: current_term: 1 leader_uuid: "d52c06abbcb44881a4837273c1ec8e91" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:34.539860 18771 tablet_service.cc:1514] Processing DeleteTablet for tablet 23442fad97424234ab659f3c0aecd0d0 with delete_type TABLET_DATA_TOMBSTONED (TS dbe891b2f04e454db1c2b5de528e01c7 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:37726
I20240502 07:59:34.539850 19642 raft_consensus.cc:2932] T d8c026e315ac4622b77b618de5c0b09f P dbe891b2f04e454db1c2b5de528e01c7 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 6fd33d34de9c463183177f1a13d4f851 (127.3.139.66) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } attrs { replace: true } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: false } } }
I20240502 07:59:34.540186 19684 tablet_replica.cc:330] T 23442fad97424234ab659f3c0aecd0d0 P dbe891b2f04e454db1c2b5de528e01c7: stopping tablet replica
I20240502 07:59:34.540297 19684 raft_consensus.cc:2229] T 23442fad97424234ab659f3c0aecd0d0 P dbe891b2f04e454db1c2b5de528e01c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:34.540201 19089 raft_consensus.cc:2932] T d8c026e315ac4622b77b618de5c0b09f P 6fd33d34de9c463183177f1a13d4f851 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 6fd33d34de9c463183177f1a13d4f851 (127.3.139.66) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } attrs { replace: true } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: false } } }
I20240502 07:59:34.540400 19684 pending_rounds.cc:62] T 23442fad97424234ab659f3c0aecd0d0 P dbe891b2f04e454db1c2b5de528e01c7: Trying to abort 1 pending ops.
I20240502 07:59:34.540453 19684 pending_rounds.cc:68] T 23442fad97424234ab659f3c0aecd0d0 P dbe891b2f04e454db1c2b5de528e01c7: Aborting op as it isn't in flight: id { term: 1 index: 5 } timestamp: 7023152228451033088 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "23442fad97424234ab659f3c0aecd0d0" old_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } attrs { replace: true } } peers { permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: NON_VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: true } } } new_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } } peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } attrs { replace: true } } peers { permanent_uuid: "db667da4313e43e2917b5acc7e6f645d" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 33695 } attrs { promote: false } } } }
I20240502 07:59:34.540558 19684 raft_consensus.cc:2894] T 23442fad97424234ab659f3c0aecd0d0 P dbe891b2f04e454db1c2b5de528e01c7 [term 1 FOLLOWER]: Aborting config change with OpId 1.5: Aborted: Op aborted
I20240502 07:59:34.540612 19684 raft_consensus.cc:2866] T 23442fad97424234ab659f3c0aecd0d0 P dbe891b2f04e454db1c2b5de528e01c7 [term 1 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20240502 07:59:34.540655 19684 raft_consensus.cc:2258] T 23442fad97424234ab659f3c0aecd0d0 P dbe891b2f04e454db1c2b5de528e01c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:34.542366 18250 catalog_manager.cc:5484] T d8c026e315ac4622b77b618de5c0b09f P dbe891b2f04e454db1c2b5de528e01c7 reported cstate change: config changed from index 4 to 5, 6fd33d34de9c463183177f1a13d4f851 (127.3.139.66) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "dbe891b2f04e454db1c2b5de528e01c7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20240502 07:59:34.542939 19750 consensus_queue.cc:1159] T 23442fad97424234ab659f3c0aecd0d0 P dbe891b2f04e454db1c2b5de528e01c7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "dbe891b2f04e454db1c2b5de528e01c7" status { last_received { term: 1 index: 5 } last_committed_idx: 4 last_received_current_leader { term: 1 index: 5 } }
I20240502 07:59:34.543949 18397 raft_consensus.cc:2932] T d8c026e315ac4622b77b618de5c0b09f P d52c06abbcb44881a4837273c1ec8e91 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 6fd33d34de9c463183177f1a13d4f851 (127.3.139.66) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } attrs { replace: true } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: false } } }
I20240502 07:59:34.544236 19684 ts_tablet_manager.cc:1875] T 23442fad97424234ab659f3c0aecd0d0 P dbe891b2f04e454db1c2b5de528e01c7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:34.545573 18790 consensus_queue.cc:237] T d8c026e315ac4622b77b618de5c0b09f P dbe891b2f04e454db1c2b5de528e01c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: false } }
I20240502 07:59:34.545900 19684 ts_tablet_manager.cc:1888] T 23442fad97424234ab659f3c0aecd0d0 P dbe891b2f04e454db1c2b5de528e01c7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 07:59:34.545961 19684 log.cc:1196] T 23442fad97424234ab659f3c0aecd0d0 P dbe891b2f04e454db1c2b5de528e01c7: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal/wals/23442fad97424234ab659f3c0aecd0d0
I20240502 07:59:34.546422 19089 raft_consensus.cc:1261] T d8c026e315ac4622b77b618de5c0b09f P 6fd33d34de9c463183177f1a13d4f851 [term 1 FOLLOWER]: Refusing update from remote peer dbe891b2f04e454db1c2b5de528e01c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:34.546459 18241 catalog_manager.cc:4830] TS dbe891b2f04e454db1c2b5de528e01c7 (127.3.139.68:43983): tablet 23442fad97424234ab659f3c0aecd0d0 (table rebalance_test_table_2 [id=d0c3034cf9d8464bb3828569fba7b9e7]) successfully deleted
I20240502 07:59:34.547356 19430 consensus_queue.cc:1035] T d8c026e315ac4622b77b618de5c0b09f P dbe891b2f04e454db1c2b5de528e01c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:34.548691 19641 raft_consensus.cc:2932] T d8c026e315ac4622b77b618de5c0b09f P dbe891b2f04e454db1c2b5de528e01c7 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER d52c06abbcb44881a4837273c1ec8e91 (127.3.139.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: false } } }
I20240502 07:59:34.549098 19089 raft_consensus.cc:2932] T d8c026e315ac4622b77b618de5c0b09f P 6fd33d34de9c463183177f1a13d4f851 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER d52c06abbcb44881a4837273c1ec8e91 (127.3.139.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: false } } }
I20240502 07:59:34.549913 18241 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet d8c026e315ac4622b77b618de5c0b09f with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:34.551112 18250 catalog_manager.cc:5484] T d8c026e315ac4622b77b618de5c0b09f P 6fd33d34de9c463183177f1a13d4f851 reported cstate change: config changed from index 5 to 6, VOTER d52c06abbcb44881a4837273c1ec8e91 (127.3.139.65) evicted. New cstate: current_term: 1 leader_uuid: "dbe891b2f04e454db1c2b5de528e01c7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: false } } }
I20240502 07:59:34.552915 18375 tablet_service.cc:1514] Processing DeleteTablet for tablet d8c026e315ac4622b77b618de5c0b09f with delete_type TABLET_DATA_TOMBSTONED (TS d52c06abbcb44881a4837273c1ec8e91 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:44040
I20240502 07:59:34.553328 19769 tablet_replica.cc:330] T d8c026e315ac4622b77b618de5c0b09f P d52c06abbcb44881a4837273c1ec8e91: stopping tablet replica
I20240502 07:59:34.553647 19769 raft_consensus.cc:2229] T d8c026e315ac4622b77b618de5c0b09f P d52c06abbcb44881a4837273c1ec8e91 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:34.553829 19769 raft_consensus.cc:2258] T d8c026e315ac4622b77b618de5c0b09f P d52c06abbcb44881a4837273c1ec8e91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:34.554833 19769 ts_tablet_manager.cc:1875] T d8c026e315ac4622b77b618de5c0b09f P d52c06abbcb44881a4837273c1ec8e91: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:34.556856 19769 ts_tablet_manager.cc:1888] T d8c026e315ac4622b77b618de5c0b09f P d52c06abbcb44881a4837273c1ec8e91: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 07:59:34.556989 19769 log.cc:1196] T d8c026e315ac4622b77b618de5c0b09f P d52c06abbcb44881a4837273c1ec8e91: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal/wals/d8c026e315ac4622b77b618de5c0b09f
I20240502 07:59:34.557512 18242 catalog_manager.cc:4830] TS d52c06abbcb44881a4837273c1ec8e91 (127.3.139.65:40139): tablet d8c026e315ac4622b77b618de5c0b09f (table rebalance_test_table_0 [id=d0fba7f1dfcc455d96585f048abd07a2]) successfully deleted
I20240502 07:59:34.562739 18397 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0a10885caf66428880d2c862ca97f8db"
dest_uuid: "d52c06abbcb44881a4837273c1ec8e91"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:44530
I20240502 07:59:34.562839 18397 raft_consensus.cc:595] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91 [term 1 LEADER]: Received request to transfer leadership
I20240502 07:59:34.580188 18397 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0a10885caf66428880d2c862ca97f8db"
dest_uuid: "d52c06abbcb44881a4837273c1ec8e91"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:44544
I20240502 07:59:34.580282 18397 raft_consensus.cc:595] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91 [term 1 LEADER]: Received request to transfer leadership
I20240502 07:59:34.677291 19642 raft_consensus.cc:1050] T d1ee0467bf17461c9692fa887cf79cc2 P dbe891b2f04e454db1c2b5de528e01c7: attempting to promote NON_VOTER 6fd33d34de9c463183177f1a13d4f851 to VOTER
I20240502 07:59:34.677610 19642 consensus_queue.cc:237] T d1ee0467bf17461c9692fa887cf79cc2 P dbe891b2f04e454db1c2b5de528e01c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } attrs { replace: true } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: false } }
I20240502 07:59:34.678738 18397 raft_consensus.cc:1261] T d1ee0467bf17461c9692fa887cf79cc2 P d52c06abbcb44881a4837273c1ec8e91 [term 1 FOLLOWER]: Refusing update from remote peer dbe891b2f04e454db1c2b5de528e01c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:34.678874 19089 raft_consensus.cc:1261] T d1ee0467bf17461c9692fa887cf79cc2 P 6fd33d34de9c463183177f1a13d4f851 [term 1 LEARNER]: Refusing update from remote peer dbe891b2f04e454db1c2b5de528e01c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:34.679287 19642 consensus_queue.cc:1035] T d1ee0467bf17461c9692fa887cf79cc2 P dbe891b2f04e454db1c2b5de528e01c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:34.679478 19641 consensus_queue.cc:1035] T d1ee0467bf17461c9692fa887cf79cc2 P dbe891b2f04e454db1c2b5de528e01c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:34.681114 19430 raft_consensus.cc:2932] T d1ee0467bf17461c9692fa887cf79cc2 P dbe891b2f04e454db1c2b5de528e01c7 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 6fd33d34de9c463183177f1a13d4f851 (127.3.139.66) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } attrs { replace: true } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: false } } }
I20240502 07:59:34.681746 19089 raft_consensus.cc:2932] T d1ee0467bf17461c9692fa887cf79cc2 P 6fd33d34de9c463183177f1a13d4f851 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 6fd33d34de9c463183177f1a13d4f851 (127.3.139.66) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } attrs { replace: true } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: false } } }
I20240502 07:59:34.682868 18397 raft_consensus.cc:2932] T d1ee0467bf17461c9692fa887cf79cc2 P d52c06abbcb44881a4837273c1ec8e91 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 6fd33d34de9c463183177f1a13d4f851 (127.3.139.66) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } attrs { replace: true } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: false } } }
I20240502 07:59:34.683375 18249 catalog_manager.cc:5484] T d1ee0467bf17461c9692fa887cf79cc2 P dbe891b2f04e454db1c2b5de528e01c7 reported cstate change: config changed from index 4 to 5, 6fd33d34de9c463183177f1a13d4f851 (127.3.139.66) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "dbe891b2f04e454db1c2b5de528e01c7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:34.685876 18790 consensus_queue.cc:237] T d1ee0467bf17461c9692fa887cf79cc2 P dbe891b2f04e454db1c2b5de528e01c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: false } }
I20240502 07:59:34.686677 19089 raft_consensus.cc:1261] T d1ee0467bf17461c9692fa887cf79cc2 P 6fd33d34de9c463183177f1a13d4f851 [term 1 FOLLOWER]: Refusing update from remote peer dbe891b2f04e454db1c2b5de528e01c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:34.686996 19641 consensus_queue.cc:1035] T d1ee0467bf17461c9692fa887cf79cc2 P dbe891b2f04e454db1c2b5de528e01c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:34.688230 19642 raft_consensus.cc:2932] T d1ee0467bf17461c9692fa887cf79cc2 P dbe891b2f04e454db1c2b5de528e01c7 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER d52c06abbcb44881a4837273c1ec8e91 (127.3.139.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: false } } }
I20240502 07:59:34.688584 19089 raft_consensus.cc:2932] T d1ee0467bf17461c9692fa887cf79cc2 P 6fd33d34de9c463183177f1a13d4f851 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER d52c06abbcb44881a4837273c1ec8e91 (127.3.139.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: false } } }
I20240502 07:59:34.689283 18241 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet d1ee0467bf17461c9692fa887cf79cc2 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:34.690025 18251 catalog_manager.cc:5484] T d1ee0467bf17461c9692fa887cf79cc2 P dbe891b2f04e454db1c2b5de528e01c7 reported cstate change: config changed from index 5 to 6, VOTER d52c06abbcb44881a4837273c1ec8e91 (127.3.139.65) evicted. New cstate: current_term: 1 leader_uuid: "dbe891b2f04e454db1c2b5de528e01c7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:34.691867 18375 tablet_service.cc:1514] Processing DeleteTablet for tablet d1ee0467bf17461c9692fa887cf79cc2 with delete_type TABLET_DATA_TOMBSTONED (TS d52c06abbcb44881a4837273c1ec8e91 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:44040
I20240502 07:59:34.692054 19769 tablet_replica.cc:330] T d1ee0467bf17461c9692fa887cf79cc2 P d52c06abbcb44881a4837273c1ec8e91: stopping tablet replica
I20240502 07:59:34.692160 19769 raft_consensus.cc:2229] T d1ee0467bf17461c9692fa887cf79cc2 P d52c06abbcb44881a4837273c1ec8e91 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:34.692291 19769 raft_consensus.cc:2258] T d1ee0467bf17461c9692fa887cf79cc2 P d52c06abbcb44881a4837273c1ec8e91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:34.692793 19769 ts_tablet_manager.cc:1875] T d1ee0467bf17461c9692fa887cf79cc2 P d52c06abbcb44881a4837273c1ec8e91: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:34.694198 19769 ts_tablet_manager.cc:1888] T d1ee0467bf17461c9692fa887cf79cc2 P d52c06abbcb44881a4837273c1ec8e91: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 07:59:34.694265 19769 log.cc:1196] T d1ee0467bf17461c9692fa887cf79cc2 P d52c06abbcb44881a4837273c1ec8e91: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal/wals/d1ee0467bf17461c9692fa887cf79cc2
I20240502 07:59:34.694653 18242 catalog_manager.cc:4830] TS d52c06abbcb44881a4837273c1ec8e91 (127.3.139.65:40139): tablet d1ee0467bf17461c9692fa887cf79cc2 (table rebalance_test_table_1 [id=30b6f8cef2c1488fadb7fd8c388373c5]) successfully deleted
I20240502 07:59:34.794204 18397 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0a10885caf66428880d2c862ca97f8db"
dest_uuid: "d52c06abbcb44881a4837273c1ec8e91"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:44556
I20240502 07:59:34.794325 18397 raft_consensus.cc:595] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91 [term 1 LEADER]: Received request to transfer leadership
I20240502 07:59:34.802598 18397 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0a10885caf66428880d2c862ca97f8db"
dest_uuid: "d52c06abbcb44881a4837273c1ec8e91"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:44564
I20240502 07:59:34.802698 18397 raft_consensus.cc:595] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91 [term 1 LEADER]: Received request to transfer leadership
I20240502 07:59:34.889163 19610 raft_consensus.cc:979] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91: : Instructing follower dbe891b2f04e454db1c2b5de528e01c7 to start an election
I20240502 07:59:34.889293 19610 raft_consensus.cc:1067] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91 [term 1 LEADER]: Signalling peer dbe891b2f04e454db1c2b5de528e01c7 to start an election
I20240502 07:59:34.889822 18790 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0a10885caf66428880d2c862ca97f8db"
dest_uuid: "dbe891b2f04e454db1c2b5de528e01c7"
from {username='slave'} at 127.3.139.65:47871
I20240502 07:59:34.889972 18790 raft_consensus.cc:482] T 0a10885caf66428880d2c862ca97f8db P dbe891b2f04e454db1c2b5de528e01c7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:34.890034 18790 raft_consensus.cc:3037] T 0a10885caf66428880d2c862ca97f8db P dbe891b2f04e454db1c2b5de528e01c7 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:34.890836 18790 raft_consensus.cc:504] T 0a10885caf66428880d2c862ca97f8db P dbe891b2f04e454db1c2b5de528e01c7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } attrs { replace: true } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: false } }
I20240502 07:59:34.891234 18790 leader_election.cc:288] T 0a10885caf66428880d2c862ca97f8db P dbe891b2f04e454db1c2b5de528e01c7 [CANDIDATE]: Term 2 election: Requested vote from peers d52c06abbcb44881a4837273c1ec8e91 (127.3.139.65:40139), 6fd33d34de9c463183177f1a13d4f851 (127.3.139.66:35635)
I20240502 07:59:34.891549 18397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a10885caf66428880d2c862ca97f8db" candidate_uuid: "dbe891b2f04e454db1c2b5de528e01c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "d52c06abbcb44881a4837273c1ec8e91"
I20240502 07:59:34.891672 18397 raft_consensus.cc:3032] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:59:34.891721 18397 raft_consensus.cc:729] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d52c06abbcb44881a4837273c1ec8e91, State: Running, Role: LEADER
I20240502 07:59:34.891678 19089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a10885caf66428880d2c862ca97f8db" candidate_uuid: "dbe891b2f04e454db1c2b5de528e01c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "6fd33d34de9c463183177f1a13d4f851"
I20240502 07:59:34.891851 19089 raft_consensus.cc:3037] T 0a10885caf66428880d2c862ca97f8db P 6fd33d34de9c463183177f1a13d4f851 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:34.891880 18397 consensus_queue.cc:260] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } attrs { replace: true } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: false } }
I20240502 07:59:34.892050 18397 raft_consensus.cc:3037] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:34.892867 19089 raft_consensus.cc:2454] T 0a10885caf66428880d2c862ca97f8db P 6fd33d34de9c463183177f1a13d4f851 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe891b2f04e454db1c2b5de528e01c7 in term 2.
I20240502 07:59:34.892861 18397 raft_consensus.cc:2454] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe891b2f04e454db1c2b5de528e01c7 in term 2.
I20240502 07:59:34.893169 18727 leader_election.cc:302] T 0a10885caf66428880d2c862ca97f8db P dbe891b2f04e454db1c2b5de528e01c7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fd33d34de9c463183177f1a13d4f851, dbe891b2f04e454db1c2b5de528e01c7; no voters:
I20240502 07:59:34.893366 19642 raft_consensus.cc:2781] T 0a10885caf66428880d2c862ca97f8db P dbe891b2f04e454db1c2b5de528e01c7 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:34.893450 19642 raft_consensus.cc:686] T 0a10885caf66428880d2c862ca97f8db P dbe891b2f04e454db1c2b5de528e01c7 [term 2 LEADER]: Becoming Leader. State: Replica: dbe891b2f04e454db1c2b5de528e01c7, State: Running, Role: LEADER
I20240502 07:59:34.893579 19642 consensus_queue.cc:237] T 0a10885caf66428880d2c862ca97f8db P dbe891b2f04e454db1c2b5de528e01c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } attrs { replace: true } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: false } }
I20240502 07:59:34.894948 18250 catalog_manager.cc:5484] T 0a10885caf66428880d2c862ca97f8db P dbe891b2f04e454db1c2b5de528e01c7 reported cstate change: term changed from 1 to 2, leader changed from d52c06abbcb44881a4837273c1ec8e91 (127.3.139.65) to dbe891b2f04e454db1c2b5de528e01c7 (127.3.139.68). New cstate: current_term: 2 leader_uuid: "dbe891b2f04e454db1c2b5de528e01c7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:35.424427 19089 raft_consensus.cc:1261] T 0a10885caf66428880d2c862ca97f8db P 6fd33d34de9c463183177f1a13d4f851 [term 2 FOLLOWER]: Refusing update from remote peer dbe891b2f04e454db1c2b5de528e01c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 07:59:35.424798 19642 consensus_queue.cc:1035] T 0a10885caf66428880d2c862ca97f8db P dbe891b2f04e454db1c2b5de528e01c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:35.427186 18397 raft_consensus.cc:1261] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91 [term 2 FOLLOWER]: Refusing update from remote peer dbe891b2f04e454db1c2b5de528e01c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 07:59:35.427706 19845 consensus_queue.cc:1035] T 0a10885caf66428880d2c862ca97f8db P dbe891b2f04e454db1c2b5de528e01c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d52c06abbcb44881a4837273c1ec8e91" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 40139 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:35.428472 18790 consensus_queue.cc:237] T 0a10885caf66428880d2c862ca97f8db P dbe891b2f04e454db1c2b5de528e01c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: false } }
I20240502 07:59:35.429545 19089 raft_consensus.cc:1261] T 0a10885caf66428880d2c862ca97f8db P 6fd33d34de9c463183177f1a13d4f851 [term 2 FOLLOWER]: Refusing update from remote peer dbe891b2f04e454db1c2b5de528e01c7: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240502 07:59:35.429864 19845 consensus_queue.cc:1035] T 0a10885caf66428880d2c862ca97f8db P dbe891b2f04e454db1c2b5de528e01c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:35.431427 19847 raft_consensus.cc:2932] T 0a10885caf66428880d2c862ca97f8db P dbe891b2f04e454db1c2b5de528e01c7 [term 2 LEADER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER d52c06abbcb44881a4837273c1ec8e91 (127.3.139.65) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: false } } }
I20240502 07:59:35.431798 19089 raft_consensus.cc:2932] T 0a10885caf66428880d2c862ca97f8db P 6fd33d34de9c463183177f1a13d4f851 [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER d52c06abbcb44881a4837273c1ec8e91 (127.3.139.65) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: false } } }
I20240502 07:59:35.432615 18241 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 0a10885caf66428880d2c862ca97f8db with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:35.433372 18250 catalog_manager.cc:5484] T 0a10885caf66428880d2c862ca97f8db P dbe891b2f04e454db1c2b5de528e01c7 reported cstate change: config changed from index 5 to 7, VOTER d52c06abbcb44881a4837273c1ec8e91 (127.3.139.65) evicted. New cstate: current_term: 2 leader_uuid: "dbe891b2f04e454db1c2b5de528e01c7" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "dbe891b2f04e454db1c2b5de528e01c7" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 43983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fd33d34de9c463183177f1a13d4f851" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 35635 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:35.435648 18375 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a10885caf66428880d2c862ca97f8db with delete_type TABLET_DATA_TOMBSTONED (TS d52c06abbcb44881a4837273c1ec8e91 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:44040
I20240502 07:59:35.435922 19854 tablet_replica.cc:330] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91: stopping tablet replica
I20240502 07:59:35.436043 19854 raft_consensus.cc:2229] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:35.436162 19854 raft_consensus.cc:2258] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:35.437124 19854 ts_tablet_manager.cc:1875] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:35.438792 19854 ts_tablet_manager.cc:1888] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.6
I20240502 07:59:35.438853 19854 log.cc:1196] T 0a10885caf66428880d2c862ca97f8db P d52c06abbcb44881a4837273c1ec8e91: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal/wals/0a10885caf66428880d2c862ca97f8db
I20240502 07:59:35.439241 18242 catalog_manager.cc:4830] TS d52c06abbcb44881a4837273c1ec8e91 (127.3.139.65:40139): tablet 0a10885caf66428880d2c862ca97f8db (table rebalance_test_table_2 [id=d0c3034cf9d8464bb3828569fba7b9e7]) successfully deleted
I20240502 07:59:35.672900 18771 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:59:35.674921 19200 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:35.679195 18375 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:35.690256 19069 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:35.765760 19069 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:35.766083 18375 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:35.766888 19200 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:35.772780 18771 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:59:35.836483 19069 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:35.838213 18375 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:35.838456 18771 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:59:35.843109 19200 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:35.874354 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 18315
I20240502 07:59:35.880013 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 19007
I20240502 07:59:35.884608 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 19138
I20240502 07:59:35.888932 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 18710
I20240502 07:59:35.893608 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 18224
2024-05-02T07:59:35Z chronyd exiting
[ OK ] LocationAwareRebalancingParamTest.Rebalance/idx7_rf2_t3_l_1_2_1 (10118 ms)
[ RUN ] LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer/idx6_rf7_t7_l_3_4_3
I20240502 07:59:35.916158 3629 test_util.cc:268] Using random seed: -67992965
I20240502 07:59:35.916869 3629 ts_itest-base.cc:112] Starting cluster with:
I20240502 07:59:35.916904 3629 ts_itest-base.cc:113] --------------
I20240502 07:59:35.916918 3629 ts_itest-base.cc:114] 10 tablet servers
I20240502 07:59:35.916931 3629 ts_itest-base.cc:115] 7 replicas per TS
I20240502 07:59:35.916944 3629 ts_itest-base.cc:116] --------------
2024-05-02T07:59:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-02T07:59:35Z Disabled control of system clock
I20240502 07:59:35.924091 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.139.126:42555
--webserver_interface=127.3.139.126
--webserver_port=0
--builtin_ntp_servers=127.3.139.84:37307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.139.126:42555
--location_mapping_cmd=/tmp/dist-test-taskcXDMw5/build/debug/bin/testdata/assign-location.py --state_store=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/location-assignment.state --map /A:3 --map /B:4 --map /C:3
--tserver_unresponsive_timeout_ms=3000
--enable_range_replica_placement=false
--raft_prepare_replacement_before_eviction=true
--enable_range_replica_placement=false
--master_client_location_assignment_enabled=false
--allow_unsafe_replication_factor with env {}
W20240502 07:59:36.021924 19970 flags.cc:425] Enabled unsafe flag: --allow_unsafe_replication_factor=true
W20240502 07:59:36.022136 19970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:36.022173 19970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:59:36.022214 19970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:36.025120 19970 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:59:36.025180 19970 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240502 07:59:36.025206 19970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:59:36.025229 19970 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240502 07:59:36.025245 19970 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240502 07:59:36.028797 19970 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:37307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal
--allow_unsafe_replication_factor=true
--enable_range_replica_placement=false
--location_mapping_cmd=/tmp/dist-test-taskcXDMw5/build/debug/bin/testdata/assign-location.py --state_store=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/location-assignment.state --map /A:3 --map /B:4 --map /C:3
--ipki_ca_key_size=768
--master_addresses=127.3.139.126:42555
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.139.126:42555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.3.139.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.19970
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:59:36.029534 19970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:59:36.030539 19970 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:59:36.037432 19976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:36.037708 19975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:36.038048 19978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:36.038488 19970 server_base.cc:1034] running on GCE node
I20240502 07:59:36.038924 19970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:59:36.039806 19970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:59:36.040956 19970 hybrid_clock.cc:648] HybridClock initialized: now 1714636776040946 us; error 23 us; skew 500 ppm
I20240502 07:59:36.042631 19970 webserver.cc:460] Webserver started at http://127.3.139.126:35631/ using document root <none> and password file <none>
I20240502 07:59:36.043182 19970 fs_manager.cc:359] Metadata directory not provided
I20240502 07:59:36.043242 19970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:59:36.043406 19970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:59:36.044932 19970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "79ece7dbeac5433883789693f01e9452"
format_stamp: "Formatted at 2024-05-02 07:59:36 on dist-test-slave-twcr"
I20240502 07:59:36.045373 19970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "79ece7dbeac5433883789693f01e9452"
format_stamp: "Formatted at 2024-05-02 07:59:36 on dist-test-slave-twcr"
I20240502 07:59:36.048372 19970 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 07:59:36.050261 19984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:36.051196 19970 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240502 07:59:36.051303 19970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal
uuid: "79ece7dbeac5433883789693f01e9452"
format_stamp: "Formatted at 2024-05-02 07:59:36 on dist-test-slave-twcr"
I20240502 07:59:36.051375 19970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:59:36.089179 19970 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:59:36.089769 19970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:59:36.089915 19970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:59:36.095821 19970 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.126:42555
I20240502 07:59:36.095825 20036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.126:42555 every 8 connection(s)
I20240502 07:59:36.097065 19970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240502 07:59:36.098191 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 19970
I20240502 07:59:36.098351 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240502 07:59:36.099911 20037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:36.105291 20037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79ece7dbeac5433883789693f01e9452: Bootstrap starting.
I20240502 07:59:36.107388 20037 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79ece7dbeac5433883789693f01e9452: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:36.108023 20037 log.cc:824] T 00000000000000000000000000000000 P 79ece7dbeac5433883789693f01e9452: Log is configured to *not* fsync() on all Append() calls
I20240502 07:59:36.109342 20037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79ece7dbeac5433883789693f01e9452: No bootstrap required, opened a new log
I20240502 07:59:36.111533 20037 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79ece7dbeac5433883789693f01e9452 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ece7dbeac5433883789693f01e9452" member_type: VOTER last_known_addr { host: "127.3.139.126" port: 42555 } }
I20240502 07:59:36.111687 20037 raft_consensus.cc:374] T 00000000000000000000000000000000 P 79ece7dbeac5433883789693f01e9452 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:36.111725 20037 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79ece7dbeac5433883789693f01e9452 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79ece7dbeac5433883789693f01e9452, State: Initialized, Role: FOLLOWER
I20240502 07:59:36.112061 20037 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79ece7dbeac5433883789693f01e9452 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ece7dbeac5433883789693f01e9452" member_type: VOTER last_known_addr { host: "127.3.139.126" port: 42555 } }
I20240502 07:59:36.112165 20037 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79ece7dbeac5433883789693f01e9452 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 07:59:36.112218 20037 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79ece7dbeac5433883789693f01e9452 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 07:59:36.112319 20037 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79ece7dbeac5433883789693f01e9452 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:36.113241 20037 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79ece7dbeac5433883789693f01e9452 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ece7dbeac5433883789693f01e9452" member_type: VOTER last_known_addr { host: "127.3.139.126" port: 42555 } }
I20240502 07:59:36.113502 20037 leader_election.cc:302] T 00000000000000000000000000000000 P 79ece7dbeac5433883789693f01e9452 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79ece7dbeac5433883789693f01e9452; no voters:
I20240502 07:59:36.113699 20037 leader_election.cc:288] T 00000000000000000000000000000000 P 79ece7dbeac5433883789693f01e9452 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 07:59:36.113794 20042 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79ece7dbeac5433883789693f01e9452 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:36.113968 20042 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79ece7dbeac5433883789693f01e9452 [term 1 LEADER]: Becoming Leader. State: Replica: 79ece7dbeac5433883789693f01e9452, State: Running, Role: LEADER
I20240502 07:59:36.114269 20042 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79ece7dbeac5433883789693f01e9452 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ece7dbeac5433883789693f01e9452" member_type: VOTER last_known_addr { host: "127.3.139.126" port: 42555 } }
I20240502 07:59:36.114567 20037 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79ece7dbeac5433883789693f01e9452 [sys.catalog]: configured and running, proceeding with master startup.
I20240502 07:59:36.115661 20044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79ece7dbeac5433883789693f01e9452 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79ece7dbeac5433883789693f01e9452. Latest consensus state: current_term: 1 leader_uuid: "79ece7dbeac5433883789693f01e9452" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ece7dbeac5433883789693f01e9452" member_type: VOTER last_known_addr { host: "127.3.139.126" port: 42555 } } }
I20240502 07:59:36.115710 20043 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79ece7dbeac5433883789693f01e9452 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "79ece7dbeac5433883789693f01e9452" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ece7dbeac5433883789693f01e9452" member_type: VOTER last_known_addr { host: "127.3.139.126" port: 42555 } } }
I20240502 07:59:36.115782 20044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79ece7dbeac5433883789693f01e9452 [sys.catalog]: This master's current role is: LEADER
I20240502 07:59:36.115809 20043 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79ece7dbeac5433883789693f01e9452 [sys.catalog]: This master's current role is: LEADER
I20240502 07:59:36.116117 20051 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240502 07:59:36.118515 20051 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240502 07:59:36.123019 20051 catalog_manager.cc:1347] Generated new cluster ID: 56f9ac362f674c15885a96da3051f507
I20240502 07:59:36.123133 20051 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240502 07:59:36.156862 20051 catalog_manager.cc:1370] Generated new certificate authority record
I20240502 07:59:36.157723 20051 catalog_manager.cc:1504] Loading token signing keys...
I20240502 07:59:36.164980 20051 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 79ece7dbeac5433883789693f01e9452: Generated new TSK 0
I20240502 07:59:36.165717 20051 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240502 07:59:36.173206 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.65:0
--local_ip_for_outbound_sockets=127.3.139.65
--webserver_interface=127.3.139.65
--webserver_port=0
--tserver_master_addrs=127.3.139.126:42555
--builtin_ntp_servers=127.3.139.84:37307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240502 07:59:36.275689 20061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:36.275871 20061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:59:36.275911 20061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:36.278795 20061 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:59:36.278849 20061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:59:36.278910 20061 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.65
I20240502 07:59:36.282301 20061 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:37307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.3.139.65
--webserver_port=0
--tserver_master_addrs=127.3.139.126:42555
--never_fsync=true
--heap_profile_path=/tmp/kudu.20061
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.65
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:59:36.283064 20061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:59:36.284057 20061 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:59:36.291491 20069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:36.291853 20066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:36.292143 20067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:36.292572 20061 server_base.cc:1034] running on GCE node
I20240502 07:59:36.292928 20061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:59:36.293440 20061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:59:36.294584 20061 hybrid_clock.cc:648] HybridClock initialized: now 1714636776294573 us; error 23 us; skew 500 ppm
I20240502 07:59:36.296286 20061 webserver.cc:460] Webserver started at http://127.3.139.65:45301/ using document root <none> and password file <none>
I20240502 07:59:36.296797 20061 fs_manager.cc:359] Metadata directory not provided
I20240502 07:59:36.296854 20061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:59:36.297013 20061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:59:36.298528 20061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "29a7180fe4574c09a8dce7c27500aab2"
format_stamp: "Formatted at 2024-05-02 07:59:36 on dist-test-slave-twcr"
I20240502 07:59:36.298975 20061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "29a7180fe4574c09a8dce7c27500aab2"
format_stamp: "Formatted at 2024-05-02 07:59:36 on dist-test-slave-twcr"
I20240502 07:59:36.302000 20061 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:36.303938 20075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:36.304847 20061 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:36.304944 20061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "29a7180fe4574c09a8dce7c27500aab2"
format_stamp: "Formatted at 2024-05-02 07:59:36 on dist-test-slave-twcr"
I20240502 07:59:36.305019 20061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:59:36.319579 20061 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:59:36.320144 20061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:59:36.320300 20061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:59:36.320787 20061 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:59:36.321632 20061 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:59:36.321681 20061 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:36.321732 20061 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:59:36.321760 20061 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:36.329339 20061 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.65:44703
I20240502 07:59:36.329372 20188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.65:44703 every 8 connection(s)
I20240502 07:59:36.330166 20061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240502 07:59:36.337872 20189 heartbeater.cc:347] Connected to a master server at 127.3.139.126:42555
I20240502 07:59:36.338058 20189 heartbeater.cc:464] Registering TS with master...
I20240502 07:59:36.338230 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 20061
I20240502 07:59:36.338341 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240502 07:59:36.338565 20189 heartbeater.cc:511] Master 127.3.139.126:42555 requested a full tablet report, sending...
I20240502 07:59:36.340334 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.66:0
--local_ip_for_outbound_sockets=127.3.139.66
--webserver_interface=127.3.139.66
--webserver_port=0
--tserver_master_addrs=127.3.139.126:42555
--builtin_ntp_servers=127.3.139.84:37307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 07:59:36.372238 20000 ts_manager.cc:194] Registered new tserver with Master: 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703)
I20240502 07:59:36.374006 20000 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.65:39085
W20240502 07:59:36.444186 20192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:36.444370 20192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:59:36.444411 20192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:36.447271 20192 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:59:36.447322 20192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:59:36.447384 20192 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.66
I20240502 07:59:36.450721 20192 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:37307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.3.139.66
--webserver_port=0
--tserver_master_addrs=127.3.139.126:42555
--never_fsync=true
--heap_profile_path=/tmp/kudu.20192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.66
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:59:36.451465 20192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:59:36.452442 20192 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:59:36.459955 20198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:36.460314 20199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:36.460681 20201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:36.461104 20192 server_base.cc:1034] running on GCE node
I20240502 07:59:36.461429 20192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:59:36.461889 20192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:59:36.463034 20192 hybrid_clock.cc:648] HybridClock initialized: now 1714636776463020 us; error 27 us; skew 500 ppm
I20240502 07:59:36.464695 20192 webserver.cc:460] Webserver started at http://127.3.139.66:37851/ using document root <none> and password file <none>
I20240502 07:59:36.465183 20192 fs_manager.cc:359] Metadata directory not provided
I20240502 07:59:36.465236 20192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:59:36.465392 20192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:59:36.466864 20192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "b7604c0eb3bf4a7cb260b564a94a1191"
format_stamp: "Formatted at 2024-05-02 07:59:36 on dist-test-slave-twcr"
I20240502 07:59:36.467309 20192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "b7604c0eb3bf4a7cb260b564a94a1191"
format_stamp: "Formatted at 2024-05-02 07:59:36 on dist-test-slave-twcr"
I20240502 07:59:36.470196 20192 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:36.472081 20207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:36.472944 20192 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:36.473035 20192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "b7604c0eb3bf4a7cb260b564a94a1191"
format_stamp: "Formatted at 2024-05-02 07:59:36 on dist-test-slave-twcr"
I20240502 07:59:36.473110 20192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:59:36.488240 20192 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:59:36.488723 20192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:59:36.488857 20192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:59:36.489312 20192 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:59:36.490113 20192 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:59:36.490159 20192 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:36.490206 20192 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:59:36.490239 20192 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:36.497861 20192 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.66:45331
I20240502 07:59:36.497884 20320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.66:45331 every 8 connection(s)
I20240502 07:59:36.498661 20192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240502 07:59:36.506207 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 20192
I20240502 07:59:36.506318 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240502 07:59:36.506384 20321 heartbeater.cc:347] Connected to a master server at 127.3.139.126:42555
I20240502 07:59:36.506546 20321 heartbeater.cc:464] Registering TS with master...
I20240502 07:59:36.508039 20321 heartbeater.cc:511] Master 127.3.139.126:42555 requested a full tablet report, sending...
I20240502 07:59:36.509546 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.67:0
--local_ip_for_outbound_sockets=127.3.139.67
--webserver_interface=127.3.139.67
--webserver_port=0
--tserver_master_addrs=127.3.139.126:42555
--builtin_ntp_servers=127.3.139.84:37307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 07:59:36.540104 20000 ts_manager.cc:194] Registered new tserver with Master: b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331)
I20240502 07:59:36.541014 20000 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.66:50079
W20240502 07:59:36.609230 20324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:36.609459 20324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:59:36.609503 20324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:36.612471 20324 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:59:36.612529 20324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:59:36.612593 20324 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.67
I20240502 07:59:36.616207 20324 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:37307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.3.139.67
--webserver_port=0
--tserver_master_addrs=127.3.139.126:42555
--never_fsync=true
--heap_profile_path=/tmp/kudu.20324
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.67
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:59:36.617067 20324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:59:36.618153 20324 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:59:36.626140 20330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:36.626443 20333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:36.626829 20331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:36.627243 20324 server_base.cc:1034] running on GCE node
I20240502 07:59:36.627571 20324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:59:36.628083 20324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:59:36.629220 20324 hybrid_clock.cc:648] HybridClock initialized: now 1714636776629198 us; error 35 us; skew 500 ppm
I20240502 07:59:36.630970 20324 webserver.cc:460] Webserver started at http://127.3.139.67:42169/ using document root <none> and password file <none>
I20240502 07:59:36.631500 20324 fs_manager.cc:359] Metadata directory not provided
I20240502 07:59:36.631553 20324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:59:36.631702 20324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:59:36.633260 20324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "35642aa2099c40e59bc2092604078e07"
format_stamp: "Formatted at 2024-05-02 07:59:36 on dist-test-slave-twcr"
I20240502 07:59:36.633677 20324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "35642aa2099c40e59bc2092604078e07"
format_stamp: "Formatted at 2024-05-02 07:59:36 on dist-test-slave-twcr"
I20240502 07:59:36.636873 20324 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 07:59:36.638864 20339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:36.639861 20324 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:36.639951 20324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "35642aa2099c40e59bc2092604078e07"
format_stamp: "Formatted at 2024-05-02 07:59:36 on dist-test-slave-twcr"
I20240502 07:59:36.640022 20324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:59:36.655589 20324 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:59:36.656194 20324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:59:36.656332 20324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:59:36.656822 20324 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:59:36.657727 20324 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:59:36.657773 20324 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:36.657820 20324 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:59:36.657841 20324 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:36.665779 20324 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.67:45587
I20240502 07:59:36.665809 20452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.67:45587 every 8 connection(s)
I20240502 07:59:36.666756 20324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240502 07:59:36.674726 20453 heartbeater.cc:347] Connected to a master server at 127.3.139.126:42555
I20240502 07:59:36.674960 20453 heartbeater.cc:464] Registering TS with master...
I20240502 07:59:36.675109 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 20324
I20240502 07:59:36.675215 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240502 07:59:36.675532 20453 heartbeater.cc:511] Master 127.3.139.126:42555 requested a full tablet report, sending...
I20240502 07:59:36.677141 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.68:0
--local_ip_for_outbound_sockets=127.3.139.68
--webserver_interface=127.3.139.68
--webserver_port=0
--tserver_master_addrs=127.3.139.126:42555
--builtin_ntp_servers=127.3.139.84:37307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 07:59:36.717579 20001 ts_manager.cc:194] Registered new tserver with Master: 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587)
I20240502 07:59:36.718598 20001 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.67:55093
W20240502 07:59:36.792093 20457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:36.792305 20457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:59:36.792346 20457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:36.795316 20457 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:59:36.795377 20457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:59:36.795441 20457 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.68
I20240502 07:59:36.798952 20457 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:37307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.68:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.3.139.68
--webserver_port=0
--tserver_master_addrs=127.3.139.126:42555
--never_fsync=true
--heap_profile_path=/tmp/kudu.20457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.68
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:59:36.799825 20457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:59:36.800891 20457 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:59:36.808827 20462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:36.809139 20465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:36.809537 20463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:36.809978 20457 server_base.cc:1034] running on GCE node
I20240502 07:59:36.810353 20457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:59:36.810884 20457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:59:36.812027 20457 hybrid_clock.cc:648] HybridClock initialized: now 1714636776812015 us; error 29 us; skew 500 ppm
I20240502 07:59:36.813783 20457 webserver.cc:460] Webserver started at http://127.3.139.68:32899/ using document root <none> and password file <none>
I20240502 07:59:36.814342 20457 fs_manager.cc:359] Metadata directory not provided
I20240502 07:59:36.814399 20457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:59:36.814580 20457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:59:36.816321 20457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "e951f009258e473cb57cf8773c7975d4"
format_stamp: "Formatted at 2024-05-02 07:59:36 on dist-test-slave-twcr"
I20240502 07:59:36.816785 20457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "e951f009258e473cb57cf8773c7975d4"
format_stamp: "Formatted at 2024-05-02 07:59:36 on dist-test-slave-twcr"
I20240502 07:59:36.819931 20457 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 07:59:36.821893 20471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:36.822845 20457 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:36.822942 20457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "e951f009258e473cb57cf8773c7975d4"
format_stamp: "Formatted at 2024-05-02 07:59:36 on dist-test-slave-twcr"
I20240502 07:59:36.823015 20457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:59:36.837786 20457 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:59:36.838327 20457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:59:36.838464 20457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:59:36.838949 20457 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:59:36.839836 20457 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:59:36.839881 20457 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:36.839929 20457 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:59:36.839954 20457 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:36.847692 20457 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.68:40315
I20240502 07:59:36.847712 20584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.68:40315 every 8 connection(s)
I20240502 07:59:36.848542 20457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240502 07:59:36.852682 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 20457
I20240502 07:59:36.852805 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240502 07:59:36.855012 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.69:0
--local_ip_for_outbound_sockets=127.3.139.69
--webserver_interface=127.3.139.69
--webserver_port=0
--tserver_master_addrs=127.3.139.126:42555
--builtin_ntp_servers=127.3.139.84:37307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 07:59:36.857026 20585 heartbeater.cc:347] Connected to a master server at 127.3.139.126:42555
I20240502 07:59:36.857244 20585 heartbeater.cc:464] Registering TS with master...
I20240502 07:59:36.857743 20585 heartbeater.cc:511] Master 127.3.139.126:42555 requested a full tablet report, sending...
I20240502 07:59:36.894729 20001 ts_manager.cc:194] Registered new tserver with Master: e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315)
I20240502 07:59:36.895726 20001 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.68:49597
W20240502 07:59:36.958667 20588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:36.958863 20588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:59:36.958904 20588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:36.961822 20588 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:59:36.961881 20588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:59:36.961944 20588 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.69
I20240502 07:59:36.965375 20588 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:37307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.69:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.3.139.69
--webserver_port=0
--tserver_master_addrs=127.3.139.126:42555
--never_fsync=true
--heap_profile_path=/tmp/kudu.20588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.69
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:59:36.966161 20588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:59:36.967348 20588 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:59:36.975152 20595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:36.975517 20597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:36.975857 20594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:36.976305 20588 server_base.cc:1034] running on GCE node
I20240502 07:59:36.976686 20588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:59:36.977228 20588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:59:36.978389 20588 hybrid_clock.cc:648] HybridClock initialized: now 1714636776978370 us; error 37 us; skew 500 ppm
I20240502 07:59:36.980252 20588 webserver.cc:460] Webserver started at http://127.3.139.69:34643/ using document root <none> and password file <none>
I20240502 07:59:36.980787 20588 fs_manager.cc:359] Metadata directory not provided
I20240502 07:59:36.980844 20588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:59:36.981010 20588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:59:36.982601 20588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "b443e68b6212450286869124684bdfff"
format_stamp: "Formatted at 2024-05-02 07:59:36 on dist-test-slave-twcr"
I20240502 07:59:36.983094 20588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "b443e68b6212450286869124684bdfff"
format_stamp: "Formatted at 2024-05-02 07:59:36 on dist-test-slave-twcr"
I20240502 07:59:36.986198 20588 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:36.988273 20603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:36.989231 20588 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:36.989331 20588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "b443e68b6212450286869124684bdfff"
format_stamp: "Formatted at 2024-05-02 07:59:36 on dist-test-slave-twcr"
I20240502 07:59:36.989406 20588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:59:37.010496 20588 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:59:37.011113 20588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:59:37.011257 20588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:59:37.011773 20588 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:59:37.012621 20588 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:59:37.012667 20588 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:37.012725 20588 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:59:37.012751 20588 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:37.020876 20588 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.69:37923
I20240502 07:59:37.020902 20716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.69:37923 every 8 connection(s)
I20240502 07:59:37.021736 20588 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240502 07:59:37.029798 20717 heartbeater.cc:347] Connected to a master server at 127.3.139.126:42555
I20240502 07:59:37.029992 20717 heartbeater.cc:464] Registering TS with master...
I20240502 07:59:37.030485 20717 heartbeater.cc:511] Master 127.3.139.126:42555 requested a full tablet report, sending...
I20240502 07:59:37.030622 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 20588
I20240502 07:59:37.030728 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20240502 07:59:37.034387 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.70:0
--local_ip_for_outbound_sockets=127.3.139.70
--webserver_interface=127.3.139.70
--webserver_port=0
--tserver_master_addrs=127.3.139.126:42555
--builtin_ntp_servers=127.3.139.84:37307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 07:59:37.062585 20001 ts_manager.cc:194] Registered new tserver with Master: b443e68b6212450286869124684bdfff (127.3.139.69:37923)
I20240502 07:59:37.063323 20001 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.69:56897
W20240502 07:59:37.141099 20721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:37.141299 20721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:59:37.141346 20721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:37.144393 20721 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:59:37.144459 20721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:59:37.144541 20721 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.70
I20240502 07:59:37.148051 20721 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:37307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.70:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.3.139.70
--webserver_port=0
--tserver_master_addrs=127.3.139.126:42555
--never_fsync=true
--heap_profile_path=/tmp/kudu.20721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.70
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:59:37.148870 20721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:59:37.149911 20721 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:59:37.157687 20727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:37.158027 20729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:37.158366 20726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:37.158792 20721 server_base.cc:1034] running on GCE node
I20240502 07:59:37.159179 20721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:59:37.159731 20721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:59:37.160887 20721 hybrid_clock.cc:648] HybridClock initialized: now 1714636777160874 us; error 24 us; skew 500 ppm
I20240502 07:59:37.162609 20721 webserver.cc:460] Webserver started at http://127.3.139.70:42307/ using document root <none> and password file <none>
I20240502 07:59:37.163177 20721 fs_manager.cc:359] Metadata directory not provided
I20240502 07:59:37.163239 20721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:59:37.163410 20721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:59:37.164973 20721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/data/instance:
uuid: "c15e23a34059456788f9775031e27ec0"
format_stamp: "Formatted at 2024-05-02 07:59:37 on dist-test-slave-twcr"
I20240502 07:59:37.165488 20721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/wal/instance:
uuid: "c15e23a34059456788f9775031e27ec0"
format_stamp: "Formatted at 2024-05-02 07:59:37 on dist-test-slave-twcr"
I20240502 07:59:37.168529 20721 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 07:59:37.170540 20735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:37.171530 20721 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:37.171638 20721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "c15e23a34059456788f9775031e27ec0"
format_stamp: "Formatted at 2024-05-02 07:59:37 on dist-test-slave-twcr"
I20240502 07:59:37.171713 20721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:59:37.182108 20721 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:59:37.182629 20721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:59:37.182775 20721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:59:37.183307 20721 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:59:37.184146 20721 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:59:37.184192 20721 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:37.184240 20721 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:59:37.184265 20721 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:37.192090 20721 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.70:36085
I20240502 07:59:37.192109 20848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.70:36085 every 8 connection(s)
I20240502 07:59:37.192991 20721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240502 07:59:37.199297 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 20721
I20240502 07:59:37.199402 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/wal/instance
I20240502 07:59:37.201031 20849 heartbeater.cc:347] Connected to a master server at 127.3.139.126:42555
I20240502 07:59:37.201236 20849 heartbeater.cc:464] Registering TS with master...
I20240502 07:59:37.201524 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.71:0
--local_ip_for_outbound_sockets=127.3.139.71
--webserver_interface=127.3.139.71
--webserver_port=0
--tserver_master_addrs=127.3.139.126:42555
--builtin_ntp_servers=127.3.139.84:37307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 07:59:37.201754 20849 heartbeater.cc:511] Master 127.3.139.126:42555 requested a full tablet report, sending...
I20240502 07:59:37.235627 20001 ts_manager.cc:194] Registered new tserver with Master: c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085)
I20240502 07:59:37.236459 20001 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.70:46865
W20240502 07:59:37.305378 20852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:37.305590 20852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:59:37.305631 20852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:37.308676 20852 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:59:37.308741 20852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:59:37.308825 20852 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.71
I20240502 07:59:37.312335 20852 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:37307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.71:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.3.139.71
--webserver_port=0
--tserver_master_addrs=127.3.139.126:42555
--never_fsync=true
--heap_profile_path=/tmp/kudu.20852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.71
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:59:37.313156 20852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:59:37.314220 20852 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:59:37.322250 20858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:37.322594 20861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:37.322959 20859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:37.323360 20852 server_base.cc:1034] running on GCE node
I20240502 07:59:37.323690 20852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:59:37.324185 20852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:59:37.325347 20852 hybrid_clock.cc:648] HybridClock initialized: now 1714636777325332 us; error 29 us; skew 500 ppm
I20240502 07:59:37.327185 20852 webserver.cc:460] Webserver started at http://127.3.139.71:44285/ using document root <none> and password file <none>
I20240502 07:59:37.327695 20852 fs_manager.cc:359] Metadata directory not provided
I20240502 07:59:37.327755 20852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:59:37.327936 20852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:59:37.329537 20852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/data/instance:
uuid: "b5f471f8b5944986a7d878cf6ac550b8"
format_stamp: "Formatted at 2024-05-02 07:59:37 on dist-test-slave-twcr"
I20240502 07:59:37.329982 20852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/wal/instance:
uuid: "b5f471f8b5944986a7d878cf6ac550b8"
format_stamp: "Formatted at 2024-05-02 07:59:37 on dist-test-slave-twcr"
I20240502 07:59:37.333141 20852 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.005s
I20240502 07:59:37.335347 20867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:37.336309 20852 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20240502 07:59:37.336406 20852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "b5f471f8b5944986a7d878cf6ac550b8"
format_stamp: "Formatted at 2024-05-02 07:59:37 on dist-test-slave-twcr"
I20240502 07:59:37.336479 20852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:59:37.354300 20852 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:59:37.354902 20852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:59:37.355072 20852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:59:37.355626 20852 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:59:37.356489 20852 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:59:37.356531 20852 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:37.356575 20852 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:59:37.356603 20852 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:37.364451 20852 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.71:39577
I20240502 07:59:37.364534 20980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.71:39577 every 8 connection(s)
I20240502 07:59:37.365370 20852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240502 07:59:37.367066 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 20852
I20240502 07:59:37.367177 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/wal/instance
I20240502 07:59:37.369480 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.72:0
--local_ip_for_outbound_sockets=127.3.139.72
--webserver_interface=127.3.139.72
--webserver_port=0
--tserver_master_addrs=127.3.139.126:42555
--builtin_ntp_servers=127.3.139.84:37307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 07:59:37.374531 20981 heartbeater.cc:347] Connected to a master server at 127.3.139.126:42555
I20240502 07:59:37.374737 20981 heartbeater.cc:464] Registering TS with master...
I20240502 07:59:37.375259 20981 heartbeater.cc:511] Master 127.3.139.126:42555 requested a full tablet report, sending...
I20240502 07:59:37.376657 20189 heartbeater.cc:503] Master 127.3.139.126:42555 was elected leader, sending a full tablet report...
I20240502 07:59:37.426815 20001 ts_manager.cc:194] Registered new tserver with Master: b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577)
I20240502 07:59:37.427795 20001 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.71:57877
W20240502 07:59:37.476420 20984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:37.476627 20984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:59:37.476667 20984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:37.479699 20984 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:59:37.479761 20984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:59:37.479827 20984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.72
I20240502 07:59:37.483336 20984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:37307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.72:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.3.139.72
--webserver_port=0
--tserver_master_addrs=127.3.139.126:42555
--never_fsync=true
--heap_profile_path=/tmp/kudu.20984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.72
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:59:37.484170 20984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:59:37.485234 20984 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:59:37.493353 20993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:37.493674 20990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:37.494035 20991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:37.494446 20984 server_base.cc:1034] running on GCE node
I20240502 07:59:37.494849 20984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:59:37.495405 20984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:59:37.496551 20984 hybrid_clock.cc:648] HybridClock initialized: now 1714636777496540 us; error 30 us; skew 500 ppm
I20240502 07:59:37.498400 20984 webserver.cc:460] Webserver started at http://127.3.139.72:46689/ using document root <none> and password file <none>
I20240502 07:59:37.498932 20984 fs_manager.cc:359] Metadata directory not provided
I20240502 07:59:37.498989 20984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:59:37.499318 20984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:59:37.500977 20984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/data/instance:
uuid: "f014c4ad4c1342219a1004181ac33ed8"
format_stamp: "Formatted at 2024-05-02 07:59:37 on dist-test-slave-twcr"
I20240502 07:59:37.501423 20984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/wal/instance:
uuid: "f014c4ad4c1342219a1004181ac33ed8"
format_stamp: "Formatted at 2024-05-02 07:59:37 on dist-test-slave-twcr"
I20240502 07:59:37.504555 20984 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 07:59:37.506559 20999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:37.507531 20984 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:37.507632 20984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "f014c4ad4c1342219a1004181ac33ed8"
format_stamp: "Formatted at 2024-05-02 07:59:37 on dist-test-slave-twcr"
I20240502 07:59:37.507706 20984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:59:37.520758 20984 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:59:37.521332 20984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:59:37.521499 20984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:59:37.522022 20984 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:59:37.522893 20984 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:59:37.522946 20984 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:37.523000 20984 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:59:37.523026 20984 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:37.530745 20984 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.72:43861
I20240502 07:59:37.530884 21112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.72:43861 every 8 connection(s)
I20240502 07:59:37.531867 20984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240502 07:59:37.535291 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 20984
I20240502 07:59:37.535415 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/wal/instance
I20240502 07:59:37.537389 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.73:0
--local_ip_for_outbound_sockets=127.3.139.73
--webserver_interface=127.3.139.73
--webserver_port=0
--tserver_master_addrs=127.3.139.126:42555
--builtin_ntp_servers=127.3.139.84:37307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 07:59:37.540472 21113 heartbeater.cc:347] Connected to a master server at 127.3.139.126:42555
I20240502 07:59:37.540700 21113 heartbeater.cc:464] Registering TS with master...
I20240502 07:59:37.541317 21113 heartbeater.cc:511] Master 127.3.139.126:42555 requested a full tablet report, sending...
I20240502 07:59:37.543044 20321 heartbeater.cc:503] Master 127.3.139.126:42555 was elected leader, sending a full tablet report...
I20240502 07:59:37.572966 20001 ts_manager.cc:194] Registered new tserver with Master: f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861)
I20240502 07:59:37.573665 20001 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.72:48549
W20240502 07:59:37.641705 21116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:37.641891 21116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:59:37.641937 21116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:37.645013 21116 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:59:37.645068 21116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:59:37.645148 21116 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.73
I20240502 07:59:37.648607 21116 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:37307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.73:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/data/info.pb
--webserver_interface=127.3.139.73
--webserver_port=0
--tserver_master_addrs=127.3.139.126:42555
--never_fsync=true
--heap_profile_path=/tmp/kudu.21116
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.73
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:59:37.649417 21116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:59:37.650447 21116 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:59:37.658254 21122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:37.658603 21123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:37.658974 21125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:37.659410 21116 server_base.cc:1034] running on GCE node
I20240502 07:59:37.659739 21116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:59:37.660198 21116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:59:37.661343 21116 hybrid_clock.cc:648] HybridClock initialized: now 1714636777661334 us; error 21 us; skew 500 ppm
I20240502 07:59:37.663086 21116 webserver.cc:460] Webserver started at http://127.3.139.73:35637/ using document root <none> and password file <none>
I20240502 07:59:37.663575 21116 fs_manager.cc:359] Metadata directory not provided
I20240502 07:59:37.663623 21116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:59:37.663769 21116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:59:37.665277 21116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/data/instance:
uuid: "bfb1b3d596ef474faa229a835329cf20"
format_stamp: "Formatted at 2024-05-02 07:59:37 on dist-test-slave-twcr"
I20240502 07:59:37.665732 21116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/wal/instance:
uuid: "bfb1b3d596ef474faa229a835329cf20"
format_stamp: "Formatted at 2024-05-02 07:59:37 on dist-test-slave-twcr"
I20240502 07:59:37.668757 21116 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.001s
I20240502 07:59:37.670753 21131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:37.671689 21116 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:37.671793 21116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/wal
uuid: "bfb1b3d596ef474faa229a835329cf20"
format_stamp: "Formatted at 2024-05-02 07:59:37 on dist-test-slave-twcr"
I20240502 07:59:37.671869 21116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:59:37.685823 21116 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:59:37.686378 21116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:59:37.686524 21116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:59:37.687017 21116 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:59:37.687876 21116 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:59:37.687922 21116 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:37.687971 21116 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:59:37.687999 21116 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:37.695803 21116 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.73:38741
I20240502 07:59:37.695832 21244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.73:38741 every 8 connection(s)
I20240502 07:59:37.696630 21116 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/data/info.pb
I20240502 07:59:37.702649 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 21116
I20240502 07:59:37.702755 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/wal/instance
I20240502 07:59:37.704600 21245 heartbeater.cc:347] Connected to a master server at 127.3.139.126:42555
I20240502 07:59:37.704816 21245 heartbeater.cc:464] Registering TS with master...
I20240502 07:59:37.704914 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-9/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-9/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-9/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-9/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.74:0
--local_ip_for_outbound_sockets=127.3.139.74
--webserver_interface=127.3.139.74
--webserver_port=0
--tserver_master_addrs=127.3.139.126:42555
--builtin_ntp_servers=127.3.139.84:37307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 07:59:37.705376 21245 heartbeater.cc:511] Master 127.3.139.126:42555 requested a full tablet report, sending...
I20240502 07:59:37.720790 20453 heartbeater.cc:503] Master 127.3.139.126:42555 was elected leader, sending a full tablet report...
I20240502 07:59:37.741994 20001 ts_manager.cc:194] Registered new tserver with Master: bfb1b3d596ef474faa229a835329cf20 (127.3.139.73:38741)
I20240502 07:59:37.743005 20001 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.73:57831
W20240502 07:59:37.812433 21248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:37.812631 21248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:59:37.812672 21248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:37.815698 21248 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:59:37.815773 21248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:59:37.815840 21248 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.74
I20240502 07:59:37.819435 21248 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:37307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-9/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-9/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.74:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-9/data/info.pb
--webserver_interface=127.3.139.74
--webserver_port=0
--tserver_master_addrs=127.3.139.126:42555
--never_fsync=true
--heap_profile_path=/tmp/kudu.21248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.74
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-9/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:59:37.820302 21248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:59:37.821355 21248 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:59:37.829224 21257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:37.829540 21254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:37.829870 21255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:37.830299 21248 server_base.cc:1034] running on GCE node
I20240502 07:59:37.830626 21248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:59:37.831146 21248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:59:37.832289 21248 hybrid_clock.cc:648] HybridClock initialized: now 1714636777832276 us; error 25 us; skew 500 ppm
I20240502 07:59:37.834031 21248 webserver.cc:460] Webserver started at http://127.3.139.74:45863/ using document root <none> and password file <none>
I20240502 07:59:37.834518 21248 fs_manager.cc:359] Metadata directory not provided
I20240502 07:59:37.834568 21248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:59:37.834719 21248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:59:37.836313 21248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-9/data/instance:
uuid: "ff9f2ccccdfa48a1afbc0e371f41c662"
format_stamp: "Formatted at 2024-05-02 07:59:37 on dist-test-slave-twcr"
I20240502 07:59:37.836745 21248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-9/wal/instance:
uuid: "ff9f2ccccdfa48a1afbc0e371f41c662"
format_stamp: "Formatted at 2024-05-02 07:59:37 on dist-test-slave-twcr"
I20240502 07:59:37.839931 21248 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 07:59:37.842046 21263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:37.843027 21248 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:37.843194 21248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-9/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-9/wal
uuid: "ff9f2ccccdfa48a1afbc0e371f41c662"
format_stamp: "Formatted at 2024-05-02 07:59:37 on dist-test-slave-twcr"
I20240502 07:59:37.843277 21248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-9/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-9/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-9/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:59:37.869006 21248 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:59:37.869589 21248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:59:37.869719 21248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:59:37.870219 21248 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:59:37.871145 21248 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:59:37.871191 21248 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:37.871229 21248 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:59:37.871249 21248 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:37.879415 21248 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.74:45559
I20240502 07:59:37.879439 21376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.74:45559 every 8 connection(s)
I20240502 07:59:37.880391 21248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-9/data/info.pb
I20240502 07:59:37.880913 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 21248
I20240502 07:59:37.881032 3629 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-9/wal/instance
I20240502 07:59:37.888933 21377 heartbeater.cc:347] Connected to a master server at 127.3.139.126:42555
I20240502 07:59:37.889142 21377 heartbeater.cc:464] Registering TS with master...
I20240502 07:59:37.889634 21377 heartbeater.cc:511] Master 127.3.139.126:42555 requested a full tablet report, sending...
I20240502 07:59:37.897938 20585 heartbeater.cc:503] Master 127.3.139.126:42555 was elected leader, sending a full tablet report...
I20240502 07:59:37.922324 20001 ts_manager.cc:194] Registered new tserver with Master: ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559)
I20240502 07:59:37.922977 20001 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.74:53867
I20240502 07:59:37.930601 3629 external_mini_cluster.cc:930] 10 TS(s) registered with all masters
I20240502 07:59:37.939635 3629 external_mini_cluster.cc:930] 10 TS(s) registered with all masters
I20240502 07:59:37.939728 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 21116
I20240502 07:59:37.944955 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 21248
I20240502 07:59:37.949131 3629 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu with pid 20984
I20240502 07:59:38.065469 20717 heartbeater.cc:503] Master 127.3.139.126:42555 was elected leader, sending a full tablet report...
I20240502 07:59:38.238440 20849 heartbeater.cc:503] Master 127.3.139.126:42555 was elected leader, sending a full tablet report...
I20240502 07:59:38.429957 20981 heartbeater.cc:503] Master 127.3.139.126:42555 was elected leader, sending a full tablet report...
I20240502 07:59:41.704941 20001 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33518:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 7
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240502 07:59:41.708045 20001 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_0 in case of a server failure: 8 tablet servers would be needed, 7 are available. Consider bringing up more tablet servers.
I20240502 07:59:41.727512 20914 tablet_service.cc:1467] Processing CreateTablet for tablet a973de06330c487ba54a8a36b1e31b54 (DEFAULT_TABLE table=rebalance_test_table_0 [id=89c0cb0099e94f1aa93c3eb299e446f0]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:41.728511 20914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a973de06330c487ba54a8a36b1e31b54. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:41.728747 20518 tablet_service.cc:1467] Processing CreateTablet for tablet a973de06330c487ba54a8a36b1e31b54 (DEFAULT_TABLE table=rebalance_test_table_0 [id=89c0cb0099e94f1aa93c3eb299e446f0]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:41.729612 20518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a973de06330c487ba54a8a36b1e31b54. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:41.728747 20517 tablet_service.cc:1467] Processing CreateTablet for tablet 15bb9480e88b41219cc91168cc438c70 (DEFAULT_TABLE table=rebalance_test_table_0 [id=89c0cb0099e94f1aa93c3eb299e446f0]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:41.731374 20517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15bb9480e88b41219cc91168cc438c70. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:41.731344 20913 tablet_service.cc:1467] Processing CreateTablet for tablet 15bb9480e88b41219cc91168cc438c70 (DEFAULT_TABLE table=rebalance_test_table_0 [id=89c0cb0099e94f1aa93c3eb299e446f0]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:41.731578 20913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15bb9480e88b41219cc91168cc438c70. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:41.733162 20782 tablet_service.cc:1467] Processing CreateTablet for tablet a973de06330c487ba54a8a36b1e31b54 (DEFAULT_TABLE table=rebalance_test_table_0 [id=89c0cb0099e94f1aa93c3eb299e446f0]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:41.733987 20782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a973de06330c487ba54a8a36b1e31b54. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:41.738337 20781 tablet_service.cc:1467] Processing CreateTablet for tablet 15bb9480e88b41219cc91168cc438c70 (DEFAULT_TABLE table=rebalance_test_table_0 [id=89c0cb0099e94f1aa93c3eb299e446f0]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:41.738576 20781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15bb9480e88b41219cc91168cc438c70. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:41.739775 20519 tablet_service.cc:1467] Processing CreateTablet for tablet 04659b5e3504411987dafc49a5fa5ea7 (DEFAULT_TABLE table=rebalance_test_table_0 [id=89c0cb0099e94f1aa93c3eb299e446f0]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:41.740010 20519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04659b5e3504411987dafc49a5fa5ea7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:41.742944 20915 tablet_service.cc:1467] Processing CreateTablet for tablet 04659b5e3504411987dafc49a5fa5ea7 (DEFAULT_TABLE table=rebalance_test_table_0 [id=89c0cb0099e94f1aa93c3eb299e446f0]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:41.743374 20915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04659b5e3504411987dafc49a5fa5ea7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:41.743531 20651 tablet_service.cc:1467] Processing CreateTablet for tablet 04659b5e3504411987dafc49a5fa5ea7 (DEFAULT_TABLE table=rebalance_test_table_0 [id=89c0cb0099e94f1aa93c3eb299e446f0]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:41.744338 20651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04659b5e3504411987dafc49a5fa5ea7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:41.744858 20650 tablet_service.cc:1467] Processing CreateTablet for tablet a973de06330c487ba54a8a36b1e31b54 (DEFAULT_TABLE table=rebalance_test_table_0 [id=89c0cb0099e94f1aa93c3eb299e446f0]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:41.745083 20650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a973de06330c487ba54a8a36b1e31b54. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:41.744835 20122 tablet_service.cc:1467] Processing CreateTablet for tablet a973de06330c487ba54a8a36b1e31b54 (DEFAULT_TABLE table=rebalance_test_table_0 [id=89c0cb0099e94f1aa93c3eb299e446f0]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:41.745635 20122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a973de06330c487ba54a8a36b1e31b54. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:41.745770 20783 tablet_service.cc:1467] Processing CreateTablet for tablet 04659b5e3504411987dafc49a5fa5ea7 (DEFAULT_TABLE table=rebalance_test_table_0 [id=89c0cb0099e94f1aa93c3eb299e446f0]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:41.745995 20783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04659b5e3504411987dafc49a5fa5ea7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:41.747330 20121 tablet_service.cc:1467] Processing CreateTablet for tablet 15bb9480e88b41219cc91168cc438c70 (DEFAULT_TABLE table=rebalance_test_table_0 [id=89c0cb0099e94f1aa93c3eb299e446f0]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:41.747565 20121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15bb9480e88b41219cc91168cc438c70. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:41.748332 20649 tablet_service.cc:1467] Processing CreateTablet for tablet 15bb9480e88b41219cc91168cc438c70 (DEFAULT_TABLE table=rebalance_test_table_0 [id=89c0cb0099e94f1aa93c3eb299e446f0]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:41.748560 20649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15bb9480e88b41219cc91168cc438c70. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:41.751066 21405 tablet_bootstrap.cc:492] T 15bb9480e88b41219cc91168cc438c70 P c15e23a34059456788f9775031e27ec0: Bootstrap starting.
I20240502 07:59:41.751112 20123 tablet_service.cc:1467] Processing CreateTablet for tablet 04659b5e3504411987dafc49a5fa5ea7 (DEFAULT_TABLE table=rebalance_test_table_0 [id=89c0cb0099e94f1aa93c3eb299e446f0]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:41.751353 20123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04659b5e3504411987dafc49a5fa5ea7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:41.753500 21405 tablet_bootstrap.cc:654] T 15bb9480e88b41219cc91168cc438c70 P c15e23a34059456788f9775031e27ec0: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:41.754398 21405 log.cc:824] T 15bb9480e88b41219cc91168cc438c70 P c15e23a34059456788f9775031e27ec0: Log is configured to *not* fsync() on all Append() calls
I20240502 07:59:41.754848 21407 tablet_bootstrap.cc:492] T a973de06330c487ba54a8a36b1e31b54 P b443e68b6212450286869124684bdfff: Bootstrap starting.
I20240502 07:59:41.756570 21404 tablet_bootstrap.cc:492] T 15bb9480e88b41219cc91168cc438c70 P e951f009258e473cb57cf8773c7975d4: Bootstrap starting.
I20240502 07:59:41.758037 20255 tablet_service.cc:1467] Processing CreateTablet for tablet 04659b5e3504411987dafc49a5fa5ea7 (DEFAULT_TABLE table=rebalance_test_table_0 [id=89c0cb0099e94f1aa93c3eb299e446f0]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:41.758453 21404 tablet_bootstrap.cc:654] T 15bb9480e88b41219cc91168cc438c70 P e951f009258e473cb57cf8773c7975d4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:41.758925 20255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04659b5e3504411987dafc49a5fa5ea7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:41.759205 21404 log.cc:824] T 15bb9480e88b41219cc91168cc438c70 P e951f009258e473cb57cf8773c7975d4: Log is configured to *not* fsync() on all Append() calls
I20240502 07:59:41.759321 20254 tablet_service.cc:1467] Processing CreateTablet for tablet a973de06330c487ba54a8a36b1e31b54 (DEFAULT_TABLE table=rebalance_test_table_0 [id=89c0cb0099e94f1aa93c3eb299e446f0]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:41.759581 20253 tablet_service.cc:1467] Processing CreateTablet for tablet 15bb9480e88b41219cc91168cc438c70 (DEFAULT_TABLE table=rebalance_test_table_0 [id=89c0cb0099e94f1aa93c3eb299e446f0]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:41.759831 20253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15bb9480e88b41219cc91168cc438c70. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:41.760823 20254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a973de06330c487ba54a8a36b1e31b54. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:41.761395 21407 tablet_bootstrap.cc:654] T a973de06330c487ba54a8a36b1e31b54 P b443e68b6212450286869124684bdfff: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:41.762133 21407 log.cc:824] T a973de06330c487ba54a8a36b1e31b54 P b443e68b6212450286869124684bdfff: Log is configured to *not* fsync() on all Append() calls
I20240502 07:59:41.762408 21405 tablet_bootstrap.cc:492] T 15bb9480e88b41219cc91168cc438c70 P c15e23a34059456788f9775031e27ec0: No bootstrap required, opened a new log
I20240502 07:59:41.762576 21405 ts_tablet_manager.cc:1367] T 15bb9480e88b41219cc91168cc438c70 P c15e23a34059456788f9775031e27ec0: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.000s
I20240502 07:59:41.762744 21408 tablet_bootstrap.cc:492] T 15bb9480e88b41219cc91168cc438c70 P 29a7180fe4574c09a8dce7c27500aab2: Bootstrap starting.
I20240502 07:59:41.764621 21408 tablet_bootstrap.cc:654] T 15bb9480e88b41219cc91168cc438c70 P 29a7180fe4574c09a8dce7c27500aab2: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:41.764741 20386 tablet_service.cc:1467] Processing CreateTablet for tablet a973de06330c487ba54a8a36b1e31b54 (DEFAULT_TABLE table=rebalance_test_table_0 [id=89c0cb0099e94f1aa93c3eb299e446f0]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:41.765367 21408 log.cc:824] T 15bb9480e88b41219cc91168cc438c70 P 29a7180fe4574c09a8dce7c27500aab2: Log is configured to *not* fsync() on all Append() calls
I20240502 07:59:41.765614 20386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a973de06330c487ba54a8a36b1e31b54. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:41.766178 21405 raft_consensus.cc:348] T 15bb9480e88b41219cc91168cc438c70 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:41.766384 21405 raft_consensus.cc:374] T 15bb9480e88b41219cc91168cc438c70 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:41.766431 21405 raft_consensus.cc:729] T 15bb9480e88b41219cc91168cc438c70 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c15e23a34059456788f9775031e27ec0, State: Initialized, Role: FOLLOWER
I20240502 07:59:41.767238 21405 consensus_queue.cc:260] T 15bb9480e88b41219cc91168cc438c70 P c15e23a34059456788f9775031e27ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:41.767349 20385 tablet_service.cc:1467] Processing CreateTablet for tablet 15bb9480e88b41219cc91168cc438c70 (DEFAULT_TABLE table=rebalance_test_table_0 [id=89c0cb0099e94f1aa93c3eb299e446f0]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:41.767570 20385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15bb9480e88b41219cc91168cc438c70. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:41.768079 21405 ts_tablet_manager.cc:1398] T 15bb9480e88b41219cc91168cc438c70 P c15e23a34059456788f9775031e27ec0: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20240502 07:59:41.768402 21405 tablet_bootstrap.cc:492] T 04659b5e3504411987dafc49a5fa5ea7 P c15e23a34059456788f9775031e27ec0: Bootstrap starting.
I20240502 07:59:41.768885 21404 tablet_bootstrap.cc:492] T 15bb9480e88b41219cc91168cc438c70 P e951f009258e473cb57cf8773c7975d4: No bootstrap required, opened a new log
I20240502 07:59:41.769057 21404 ts_tablet_manager.cc:1367] T 15bb9480e88b41219cc91168cc438c70 P e951f009258e473cb57cf8773c7975d4: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.004s
I20240502 07:59:41.769248 21405 tablet_bootstrap.cc:654] T 04659b5e3504411987dafc49a5fa5ea7 P c15e23a34059456788f9775031e27ec0: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:41.769608 21407 tablet_bootstrap.cc:492] T a973de06330c487ba54a8a36b1e31b54 P b443e68b6212450286869124684bdfff: No bootstrap required, opened a new log
I20240502 07:59:41.769665 20387 tablet_service.cc:1467] Processing CreateTablet for tablet 04659b5e3504411987dafc49a5fa5ea7 (DEFAULT_TABLE table=rebalance_test_table_0 [id=89c0cb0099e94f1aa93c3eb299e446f0]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:41.769794 21407 ts_tablet_manager.cc:1367] T a973de06330c487ba54a8a36b1e31b54 P b443e68b6212450286869124684bdfff: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.000s
I20240502 07:59:41.769889 20387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04659b5e3504411987dafc49a5fa5ea7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:41.770234 21405 tablet_bootstrap.cc:492] T 04659b5e3504411987dafc49a5fa5ea7 P c15e23a34059456788f9775031e27ec0: No bootstrap required, opened a new log
I20240502 07:59:41.770309 21405 ts_tablet_manager.cc:1367] T 04659b5e3504411987dafc49a5fa5ea7 P c15e23a34059456788f9775031e27ec0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:41.770787 21405 raft_consensus.cc:348] T 04659b5e3504411987dafc49a5fa5ea7 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } }
I20240502 07:59:41.770888 21405 raft_consensus.cc:374] T 04659b5e3504411987dafc49a5fa5ea7 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:41.770915 21405 raft_consensus.cc:729] T 04659b5e3504411987dafc49a5fa5ea7 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c15e23a34059456788f9775031e27ec0, State: Initialized, Role: FOLLOWER
I20240502 07:59:41.771020 21405 consensus_queue.cc:260] T 04659b5e3504411987dafc49a5fa5ea7 P c15e23a34059456788f9775031e27ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } }
I20240502 07:59:41.771390 21405 ts_tablet_manager.cc:1398] T 04659b5e3504411987dafc49a5fa5ea7 P c15e23a34059456788f9775031e27ec0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:41.771540 21405 tablet_bootstrap.cc:492] T a973de06330c487ba54a8a36b1e31b54 P c15e23a34059456788f9775031e27ec0: Bootstrap starting.
I20240502 07:59:41.772367 21405 tablet_bootstrap.cc:654] T a973de06330c487ba54a8a36b1e31b54 P c15e23a34059456788f9775031e27ec0: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:41.772459 21404 raft_consensus.cc:348] T 15bb9480e88b41219cc91168cc438c70 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:41.772661 21404 raft_consensus.cc:374] T 15bb9480e88b41219cc91168cc438c70 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:41.772706 21404 raft_consensus.cc:729] T 15bb9480e88b41219cc91168cc438c70 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e951f009258e473cb57cf8773c7975d4, State: Initialized, Role: FOLLOWER
I20240502 07:59:41.772804 21407 raft_consensus.cc:348] T a973de06330c487ba54a8a36b1e31b54 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:41.773016 21407 raft_consensus.cc:374] T a973de06330c487ba54a8a36b1e31b54 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:41.773070 21407 raft_consensus.cc:729] T a973de06330c487ba54a8a36b1e31b54 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b443e68b6212450286869124684bdfff, State: Initialized, Role: FOLLOWER
I20240502 07:59:41.773145 21404 consensus_queue.cc:260] T 15bb9480e88b41219cc91168cc438c70 P e951f009258e473cb57cf8773c7975d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:41.773442 21407 consensus_queue.cc:260] T a973de06330c487ba54a8a36b1e31b54 P b443e68b6212450286869124684bdfff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:41.774048 21404 ts_tablet_manager.cc:1398] T 15bb9480e88b41219cc91168cc438c70 P e951f009258e473cb57cf8773c7975d4: Time spent starting tablet: real 0.005s user 0.003s sys 0.002s
I20240502 07:59:41.774148 21407 ts_tablet_manager.cc:1398] T a973de06330c487ba54a8a36b1e31b54 P b443e68b6212450286869124684bdfff: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20240502 07:59:41.774477 21404 tablet_bootstrap.cc:492] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4: Bootstrap starting.
W20240502 07:59:41.775635 20718 tablet.cc:2369] T a973de06330c487ba54a8a36b1e31b54 P b443e68b6212450286869124684bdfff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 07:59:41.775929 21404 tablet_bootstrap.cc:654] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:41.776532 21407 tablet_bootstrap.cc:492] T 04659b5e3504411987dafc49a5fa5ea7 P b443e68b6212450286869124684bdfff: Bootstrap starting.
I20240502 07:59:41.777391 21407 tablet_bootstrap.cc:654] T 04659b5e3504411987dafc49a5fa5ea7 P b443e68b6212450286869124684bdfff: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:41.778419 21407 tablet_bootstrap.cc:492] T 04659b5e3504411987dafc49a5fa5ea7 P b443e68b6212450286869124684bdfff: No bootstrap required, opened a new log
I20240502 07:59:41.778420 21413 tablet_bootstrap.cc:492] T 04659b5e3504411987dafc49a5fa5ea7 P b7604c0eb3bf4a7cb260b564a94a1191: Bootstrap starting.
I20240502 07:59:41.778556 21407 ts_tablet_manager.cc:1367] T 04659b5e3504411987dafc49a5fa5ea7 P b443e68b6212450286869124684bdfff: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:41.779110 21407 raft_consensus.cc:348] T 04659b5e3504411987dafc49a5fa5ea7 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } }
I20240502 07:59:41.779301 21407 raft_consensus.cc:374] T 04659b5e3504411987dafc49a5fa5ea7 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:41.779369 21407 raft_consensus.cc:729] T 04659b5e3504411987dafc49a5fa5ea7 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b443e68b6212450286869124684bdfff, State: Initialized, Role: FOLLOWER
I20240502 07:59:41.779552 21407 consensus_queue.cc:260] T 04659b5e3504411987dafc49a5fa5ea7 P b443e68b6212450286869124684bdfff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } }
I20240502 07:59:41.779961 21407 ts_tablet_manager.cc:1398] T 04659b5e3504411987dafc49a5fa5ea7 P b443e68b6212450286869124684bdfff: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:41.780073 21406 tablet_bootstrap.cc:492] T 04659b5e3504411987dafc49a5fa5ea7 P b5f471f8b5944986a7d878cf6ac550b8: Bootstrap starting.
I20240502 07:59:41.780478 21413 tablet_bootstrap.cc:654] T 04659b5e3504411987dafc49a5fa5ea7 P b7604c0eb3bf4a7cb260b564a94a1191: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:41.780771 21407 tablet_bootstrap.cc:492] T 15bb9480e88b41219cc91168cc438c70 P b443e68b6212450286869124684bdfff: Bootstrap starting.
I20240502 07:59:41.781236 21413 log.cc:824] T 04659b5e3504411987dafc49a5fa5ea7 P b7604c0eb3bf4a7cb260b564a94a1191: Log is configured to *not* fsync() on all Append() calls
I20240502 07:59:41.781271 21422 tablet_bootstrap.cc:492] T a973de06330c487ba54a8a36b1e31b54 P 35642aa2099c40e59bc2092604078e07: Bootstrap starting.
I20240502 07:59:41.781987 21407 tablet_bootstrap.cc:654] T 15bb9480e88b41219cc91168cc438c70 P b443e68b6212450286869124684bdfff: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:41.782539 21406 tablet_bootstrap.cc:654] T 04659b5e3504411987dafc49a5fa5ea7 P b5f471f8b5944986a7d878cf6ac550b8: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:41.782698 21408 tablet_bootstrap.cc:492] T 15bb9480e88b41219cc91168cc438c70 P 29a7180fe4574c09a8dce7c27500aab2: No bootstrap required, opened a new log
I20240502 07:59:41.782886 21408 ts_tablet_manager.cc:1367] T 15bb9480e88b41219cc91168cc438c70 P 29a7180fe4574c09a8dce7c27500aab2: Time spent bootstrapping tablet: real 0.020s user 0.003s sys 0.000s
I20240502 07:59:41.782991 21422 tablet_bootstrap.cc:654] T a973de06330c487ba54a8a36b1e31b54 P 35642aa2099c40e59bc2092604078e07: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:41.783293 21406 log.cc:824] T 04659b5e3504411987dafc49a5fa5ea7 P b5f471f8b5944986a7d878cf6ac550b8: Log is configured to *not* fsync() on all Append() calls
I20240502 07:59:41.783910 21422 log.cc:824] T a973de06330c487ba54a8a36b1e31b54 P 35642aa2099c40e59bc2092604078e07: Log is configured to *not* fsync() on all Append() calls
I20240502 07:59:41.784997 21405 tablet_bootstrap.cc:492] T a973de06330c487ba54a8a36b1e31b54 P c15e23a34059456788f9775031e27ec0: No bootstrap required, opened a new log
I20240502 07:59:41.785082 21405 ts_tablet_manager.cc:1367] T a973de06330c487ba54a8a36b1e31b54 P c15e23a34059456788f9775031e27ec0: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.000s
I20240502 07:59:41.785198 21404 tablet_bootstrap.cc:492] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4: No bootstrap required, opened a new log
I20240502 07:59:41.785274 21404 ts_tablet_manager.cc:1367] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 07:59:41.785480 21405 raft_consensus.cc:348] T a973de06330c487ba54a8a36b1e31b54 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:41.785590 21405 raft_consensus.cc:374] T a973de06330c487ba54a8a36b1e31b54 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:41.785617 21405 raft_consensus.cc:729] T a973de06330c487ba54a8a36b1e31b54 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c15e23a34059456788f9775031e27ec0, State: Initialized, Role: FOLLOWER
I20240502 07:59:41.785737 21405 consensus_queue.cc:260] T a973de06330c487ba54a8a36b1e31b54 P c15e23a34059456788f9775031e27ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:41.785756 21404 raft_consensus.cc:348] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:41.785867 21404 raft_consensus.cc:374] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:41.785895 21404 raft_consensus.cc:729] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e951f009258e473cb57cf8773c7975d4, State: Initialized, Role: FOLLOWER
I20240502 07:59:41.786015 21405 ts_tablet_manager.cc:1398] T a973de06330c487ba54a8a36b1e31b54 P c15e23a34059456788f9775031e27ec0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:41.786010 21404 consensus_queue.cc:260] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:41.786262 21404 ts_tablet_manager.cc:1398] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:41.786412 21404 tablet_bootstrap.cc:492] T 04659b5e3504411987dafc49a5fa5ea7 P e951f009258e473cb57cf8773c7975d4: Bootstrap starting.
I20240502 07:59:41.786347 21408 raft_consensus.cc:348] T 15bb9480e88b41219cc91168cc438c70 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:41.786541 21408 raft_consensus.cc:374] T 15bb9480e88b41219cc91168cc438c70 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:41.786583 21408 raft_consensus.cc:729] T 15bb9480e88b41219cc91168cc438c70 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29a7180fe4574c09a8dce7c27500aab2, State: Initialized, Role: FOLLOWER
I20240502 07:59:41.786674 21406 tablet_bootstrap.cc:492] T 04659b5e3504411987dafc49a5fa5ea7 P b5f471f8b5944986a7d878cf6ac550b8: No bootstrap required, opened a new log
I20240502 07:59:41.786841 21406 ts_tablet_manager.cc:1367] T 04659b5e3504411987dafc49a5fa5ea7 P b5f471f8b5944986a7d878cf6ac550b8: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20240502 07:59:41.787024 21408 consensus_queue.cc:260] T 15bb9480e88b41219cc91168cc438c70 P 29a7180fe4574c09a8dce7c27500aab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:41.787297 21404 tablet_bootstrap.cc:654] T 04659b5e3504411987dafc49a5fa5ea7 P e951f009258e473cb57cf8773c7975d4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:41.787478 21407 tablet_bootstrap.cc:492] T 15bb9480e88b41219cc91168cc438c70 P b443e68b6212450286869124684bdfff: No bootstrap required, opened a new log
I20240502 07:59:41.787550 21407 ts_tablet_manager.cc:1367] T 15bb9480e88b41219cc91168cc438c70 P b443e68b6212450286869124684bdfff: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20240502 07:59:41.787894 21408 ts_tablet_manager.cc:1398] T 15bb9480e88b41219cc91168cc438c70 P 29a7180fe4574c09a8dce7c27500aab2: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20240502 07:59:41.788228 21408 tablet_bootstrap.cc:492] T 04659b5e3504411987dafc49a5fa5ea7 P 29a7180fe4574c09a8dce7c27500aab2: Bootstrap starting.
I20240502 07:59:41.788530 21407 raft_consensus.cc:348] T 15bb9480e88b41219cc91168cc438c70 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:41.788646 21407 raft_consensus.cc:374] T 15bb9480e88b41219cc91168cc438c70 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:41.788679 21407 raft_consensus.cc:729] T 15bb9480e88b41219cc91168cc438c70 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b443e68b6212450286869124684bdfff, State: Initialized, Role: FOLLOWER
I20240502 07:59:41.788787 21407 consensus_queue.cc:260] T 15bb9480e88b41219cc91168cc438c70 P b443e68b6212450286869124684bdfff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:41.789038 21407 ts_tablet_manager.cc:1398] T 15bb9480e88b41219cc91168cc438c70 P b443e68b6212450286869124684bdfff: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:41.789103 21408 tablet_bootstrap.cc:654] T 04659b5e3504411987dafc49a5fa5ea7 P 29a7180fe4574c09a8dce7c27500aab2: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:41.790089 21413 tablet_bootstrap.cc:492] T 04659b5e3504411987dafc49a5fa5ea7 P b7604c0eb3bf4a7cb260b564a94a1191: No bootstrap required, opened a new log
I20240502 07:59:41.790123 21408 tablet_bootstrap.cc:492] T 04659b5e3504411987dafc49a5fa5ea7 P 29a7180fe4574c09a8dce7c27500aab2: No bootstrap required, opened a new log
I20240502 07:59:41.790197 21408 ts_tablet_manager.cc:1367] T 04659b5e3504411987dafc49a5fa5ea7 P 29a7180fe4574c09a8dce7c27500aab2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:41.790241 21413 ts_tablet_manager.cc:1367] T 04659b5e3504411987dafc49a5fa5ea7 P b7604c0eb3bf4a7cb260b564a94a1191: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.000s
I20240502 07:59:41.791523 21406 raft_consensus.cc:348] T 04659b5e3504411987dafc49a5fa5ea7 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } }
I20240502 07:59:41.791760 21406 raft_consensus.cc:374] T 04659b5e3504411987dafc49a5fa5ea7 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:41.791719 21408 raft_consensus.cc:348] T 04659b5e3504411987dafc49a5fa5ea7 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } }
I20240502 07:59:41.791826 21406 raft_consensus.cc:729] T 04659b5e3504411987dafc49a5fa5ea7 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5f471f8b5944986a7d878cf6ac550b8, State: Initialized, Role: FOLLOWER
I20240502 07:59:41.791842 21408 raft_consensus.cc:374] T 04659b5e3504411987dafc49a5fa5ea7 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:41.791879 21408 raft_consensus.cc:729] T 04659b5e3504411987dafc49a5fa5ea7 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29a7180fe4574c09a8dce7c27500aab2, State: Initialized, Role: FOLLOWER
I20240502 07:59:41.792006 21408 consensus_queue.cc:260] T 04659b5e3504411987dafc49a5fa5ea7 P 29a7180fe4574c09a8dce7c27500aab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } }
I20240502 07:59:41.792344 21408 ts_tablet_manager.cc:1398] T 04659b5e3504411987dafc49a5fa5ea7 P 29a7180fe4574c09a8dce7c27500aab2: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:41.792382 21406 consensus_queue.cc:260] T 04659b5e3504411987dafc49a5fa5ea7 P b5f471f8b5944986a7d878cf6ac550b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } }
I20240502 07:59:41.793270 21406 ts_tablet_manager.cc:1398] T 04659b5e3504411987dafc49a5fa5ea7 P b5f471f8b5944986a7d878cf6ac550b8: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20240502 07:59:41.793637 21406 tablet_bootstrap.cc:492] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8: Bootstrap starting.
I20240502 07:59:41.794324 21408 tablet_bootstrap.cc:492] T a973de06330c487ba54a8a36b1e31b54 P 29a7180fe4574c09a8dce7c27500aab2: Bootstrap starting.
I20240502 07:59:41.794389 21422 tablet_bootstrap.cc:492] T a973de06330c487ba54a8a36b1e31b54 P 35642aa2099c40e59bc2092604078e07: No bootstrap required, opened a new log
I20240502 07:59:41.794538 21422 ts_tablet_manager.cc:1367] T a973de06330c487ba54a8a36b1e31b54 P 35642aa2099c40e59bc2092604078e07: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.000s
I20240502 07:59:41.794414 21413 raft_consensus.cc:348] T 04659b5e3504411987dafc49a5fa5ea7 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } }
I20240502 07:59:41.794600 21413 raft_consensus.cc:374] T 04659b5e3504411987dafc49a5fa5ea7 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:41.794662 21413 raft_consensus.cc:729] T 04659b5e3504411987dafc49a5fa5ea7 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7604c0eb3bf4a7cb260b564a94a1191, State: Initialized, Role: FOLLOWER
I20240502 07:59:41.794687 21406 tablet_bootstrap.cc:654] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:41.795073 21413 consensus_queue.cc:260] T 04659b5e3504411987dafc49a5fa5ea7 P b7604c0eb3bf4a7cb260b564a94a1191 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } }
I20240502 07:59:41.795279 21408 tablet_bootstrap.cc:654] T a973de06330c487ba54a8a36b1e31b54 P 29a7180fe4574c09a8dce7c27500aab2: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:41.795843 21413 ts_tablet_manager.cc:1398] T 04659b5e3504411987dafc49a5fa5ea7 P b7604c0eb3bf4a7cb260b564a94a1191: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20240502 07:59:41.796041 21406 tablet_bootstrap.cc:492] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8: No bootstrap required, opened a new log
I20240502 07:59:41.796114 21406 ts_tablet_manager.cc:1367] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:59:41.796149 21413 tablet_bootstrap.cc:492] T a973de06330c487ba54a8a36b1e31b54 P b7604c0eb3bf4a7cb260b564a94a1191: Bootstrap starting.
I20240502 07:59:41.796592 21406 raft_consensus.cc:348] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:41.796697 21406 raft_consensus.cc:374] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:41.796726 21406 raft_consensus.cc:729] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5f471f8b5944986a7d878cf6ac550b8, State: Initialized, Role: FOLLOWER
I20240502 07:59:41.796877 21406 consensus_queue.cc:260] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:41.796994 21413 tablet_bootstrap.cc:654] T a973de06330c487ba54a8a36b1e31b54 P b7604c0eb3bf4a7cb260b564a94a1191: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:41.797147 21406 ts_tablet_manager.cc:1398] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:41.797297 21406 tablet_bootstrap.cc:492] T a973de06330c487ba54a8a36b1e31b54 P b5f471f8b5944986a7d878cf6ac550b8: Bootstrap starting.
I20240502 07:59:41.797773 21422 raft_consensus.cc:348] T a973de06330c487ba54a8a36b1e31b54 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:41.797966 21422 raft_consensus.cc:374] T a973de06330c487ba54a8a36b1e31b54 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:41.798027 21422 raft_consensus.cc:729] T a973de06330c487ba54a8a36b1e31b54 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35642aa2099c40e59bc2092604078e07, State: Initialized, Role: FOLLOWER
I20240502 07:59:41.798074 21406 tablet_bootstrap.cc:654] T a973de06330c487ba54a8a36b1e31b54 P b5f471f8b5944986a7d878cf6ac550b8: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:41.798170 21413 tablet_bootstrap.cc:492] T a973de06330c487ba54a8a36b1e31b54 P b7604c0eb3bf4a7cb260b564a94a1191: No bootstrap required, opened a new log
I20240502 07:59:41.798241 21413 ts_tablet_manager.cc:1367] T a973de06330c487ba54a8a36b1e31b54 P b7604c0eb3bf4a7cb260b564a94a1191: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:41.798717 21413 raft_consensus.cc:348] T a973de06330c487ba54a8a36b1e31b54 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:41.798820 21413 raft_consensus.cc:374] T a973de06330c487ba54a8a36b1e31b54 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:41.798849 21413 raft_consensus.cc:729] T a973de06330c487ba54a8a36b1e31b54 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7604c0eb3bf4a7cb260b564a94a1191, State: Initialized, Role: FOLLOWER
I20240502 07:59:41.798960 21413 consensus_queue.cc:260] T a973de06330c487ba54a8a36b1e31b54 P b7604c0eb3bf4a7cb260b564a94a1191 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:41.799088 21406 tablet_bootstrap.cc:492] T a973de06330c487ba54a8a36b1e31b54 P b5f471f8b5944986a7d878cf6ac550b8: No bootstrap required, opened a new log
I20240502 07:59:41.799165 21406 ts_tablet_manager.cc:1367] T a973de06330c487ba54a8a36b1e31b54 P b5f471f8b5944986a7d878cf6ac550b8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:41.799235 21413 ts_tablet_manager.cc:1398] T a973de06330c487ba54a8a36b1e31b54 P b7604c0eb3bf4a7cb260b564a94a1191: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:41.799376 21413 tablet_bootstrap.cc:492] T 15bb9480e88b41219cc91168cc438c70 P b7604c0eb3bf4a7cb260b564a94a1191: Bootstrap starting.
I20240502 07:59:41.799542 21406 raft_consensus.cc:348] T a973de06330c487ba54a8a36b1e31b54 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:41.799660 21406 raft_consensus.cc:374] T a973de06330c487ba54a8a36b1e31b54 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:41.799691 21406 raft_consensus.cc:729] T a973de06330c487ba54a8a36b1e31b54 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5f471f8b5944986a7d878cf6ac550b8, State: Initialized, Role: FOLLOWER
I20240502 07:59:41.799803 21406 consensus_queue.cc:260] T a973de06330c487ba54a8a36b1e31b54 P b5f471f8b5944986a7d878cf6ac550b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:41.800050 21406 ts_tablet_manager.cc:1398] T a973de06330c487ba54a8a36b1e31b54 P b5f471f8b5944986a7d878cf6ac550b8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:41.800228 21413 tablet_bootstrap.cc:654] T 15bb9480e88b41219cc91168cc438c70 P b7604c0eb3bf4a7cb260b564a94a1191: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:41.801084 21422 consensus_queue.cc:260] T a973de06330c487ba54a8a36b1e31b54 P 35642aa2099c40e59bc2092604078e07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:41.801362 21413 tablet_bootstrap.cc:492] T 15bb9480e88b41219cc91168cc438c70 P b7604c0eb3bf4a7cb260b564a94a1191: No bootstrap required, opened a new log
I20240502 07:59:41.801434 21413 ts_tablet_manager.cc:1367] T 15bb9480e88b41219cc91168cc438c70 P b7604c0eb3bf4a7cb260b564a94a1191: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:41.801805 21413 raft_consensus.cc:348] T 15bb9480e88b41219cc91168cc438c70 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:41.801880 21413 raft_consensus.cc:374] T 15bb9480e88b41219cc91168cc438c70 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:41.801904 21413 raft_consensus.cc:729] T 15bb9480e88b41219cc91168cc438c70 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7604c0eb3bf4a7cb260b564a94a1191, State: Initialized, Role: FOLLOWER
I20240502 07:59:41.801936 21422 ts_tablet_manager.cc:1398] T a973de06330c487ba54a8a36b1e31b54 P 35642aa2099c40e59bc2092604078e07: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20240502 07:59:41.802000 21413 consensus_queue.cc:260] T 15bb9480e88b41219cc91168cc438c70 P b7604c0eb3bf4a7cb260b564a94a1191 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:41.802244 21422 tablet_bootstrap.cc:492] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07: Bootstrap starting.
I20240502 07:59:41.802258 21413 ts_tablet_manager.cc:1398] T 15bb9480e88b41219cc91168cc438c70 P b7604c0eb3bf4a7cb260b564a94a1191: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:41.803126 21422 tablet_bootstrap.cc:654] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:41.803112 21408 tablet_bootstrap.cc:492] T a973de06330c487ba54a8a36b1e31b54 P 29a7180fe4574c09a8dce7c27500aab2: No bootstrap required, opened a new log
I20240502 07:59:41.803205 21408 ts_tablet_manager.cc:1367] T a973de06330c487ba54a8a36b1e31b54 P 29a7180fe4574c09a8dce7c27500aab2: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.001s
I20240502 07:59:41.803931 21404 tablet_bootstrap.cc:492] T 04659b5e3504411987dafc49a5fa5ea7 P e951f009258e473cb57cf8773c7975d4: No bootstrap required, opened a new log
I20240502 07:59:41.804020 21404 ts_tablet_manager.cc:1367] T 04659b5e3504411987dafc49a5fa5ea7 P e951f009258e473cb57cf8773c7975d4: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.000s
I20240502 07:59:41.804411 21408 raft_consensus.cc:348] T a973de06330c487ba54a8a36b1e31b54 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:41.804428 21404 raft_consensus.cc:348] T 04659b5e3504411987dafc49a5fa5ea7 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } }
I20240502 07:59:41.804530 21404 raft_consensus.cc:374] T 04659b5e3504411987dafc49a5fa5ea7 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:41.804562 21404 raft_consensus.cc:729] T 04659b5e3504411987dafc49a5fa5ea7 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e951f009258e473cb57cf8773c7975d4, State: Initialized, Role: FOLLOWER
I20240502 07:59:41.804677 21404 consensus_queue.cc:260] T 04659b5e3504411987dafc49a5fa5ea7 P e951f009258e473cb57cf8773c7975d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } }
I20240502 07:59:41.804739 21422 tablet_bootstrap.cc:492] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07: No bootstrap required, opened a new log
I20240502 07:59:41.804813 21422 ts_tablet_manager.cc:1367] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 07:59:41.804809 21408 raft_consensus.cc:374] T a973de06330c487ba54a8a36b1e31b54 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:41.804854 21408 raft_consensus.cc:729] T a973de06330c487ba54a8a36b1e31b54 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29a7180fe4574c09a8dce7c27500aab2, State: Initialized, Role: FOLLOWER
I20240502 07:59:41.804950 21404 ts_tablet_manager.cc:1398] T 04659b5e3504411987dafc49a5fa5ea7 P e951f009258e473cb57cf8773c7975d4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:41.804994 21408 consensus_queue.cc:260] T a973de06330c487ba54a8a36b1e31b54 P 29a7180fe4574c09a8dce7c27500aab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:41.805291 21408 ts_tablet_manager.cc:1398] T a973de06330c487ba54a8a36b1e31b54 P 29a7180fe4574c09a8dce7c27500aab2: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:41.805291 21422 raft_consensus.cc:348] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } }
I20240502 07:59:41.805394 21422 raft_consensus.cc:374] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:41.805426 21422 raft_consensus.cc:729] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35642aa2099c40e59bc2092604078e07, State: Initialized, Role: FOLLOWER
I20240502 07:59:41.805534 21422 consensus_queue.cc:260] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } }
I20240502 07:59:41.805866 21422 ts_tablet_manager.cc:1398] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:41.806012 21422 tablet_bootstrap.cc:492] T 15bb9480e88b41219cc91168cc438c70 P 35642aa2099c40e59bc2092604078e07: Bootstrap starting.
I20240502 07:59:41.806864 21422 tablet_bootstrap.cc:654] T 15bb9480e88b41219cc91168cc438c70 P 35642aa2099c40e59bc2092604078e07: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:41.807879 21422 tablet_bootstrap.cc:492] T 15bb9480e88b41219cc91168cc438c70 P 35642aa2099c40e59bc2092604078e07: No bootstrap required, opened a new log
I20240502 07:59:41.807953 21422 ts_tablet_manager.cc:1367] T 15bb9480e88b41219cc91168cc438c70 P 35642aa2099c40e59bc2092604078e07: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 07:59:41.808288 21422 raft_consensus.cc:348] T 15bb9480e88b41219cc91168cc438c70 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:41.808382 21422 raft_consensus.cc:374] T 15bb9480e88b41219cc91168cc438c70 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:41.808413 21422 raft_consensus.cc:729] T 15bb9480e88b41219cc91168cc438c70 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35642aa2099c40e59bc2092604078e07, State: Initialized, Role: FOLLOWER
I20240502 07:59:41.808529 21422 consensus_queue.cc:260] T 15bb9480e88b41219cc91168cc438c70 P 35642aa2099c40e59bc2092604078e07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:41.808804 21422 ts_tablet_manager.cc:1398] T 15bb9480e88b41219cc91168cc438c70 P 35642aa2099c40e59bc2092604078e07: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:41.830554 21441 raft_consensus.cc:482] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:41.830718 21441 raft_consensus.cc:504] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } }
I20240502 07:59:41.833297 21441 leader_election.cc:288] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577), e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315), b443e68b6212450286869124684bdfff (127.3.139.69:37923), c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085), 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703), b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331)
W20240502 07:59:41.834679 20190 tablet.cc:2369] T 04659b5e3504411987dafc49a5fa5ea7 P 29a7180fe4574c09a8dce7c27500aab2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 07:59:41.836839 20671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04659b5e3504411987dafc49a5fa5ea7" candidate_uuid: "35642aa2099c40e59bc2092604078e07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b443e68b6212450286869124684bdfff" is_pre_election: true
I20240502 07:59:41.837114 20671 raft_consensus.cc:2454] T 04659b5e3504411987dafc49a5fa5ea7 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35642aa2099c40e59bc2092604078e07 in term 0.
I20240502 07:59:41.838213 20935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04659b5e3504411987dafc49a5fa5ea7" candidate_uuid: "35642aa2099c40e59bc2092604078e07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f471f8b5944986a7d878cf6ac550b8" is_pre_election: true
I20240502 07:59:41.838376 20539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04659b5e3504411987dafc49a5fa5ea7" candidate_uuid: "35642aa2099c40e59bc2092604078e07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e951f009258e473cb57cf8773c7975d4" is_pre_election: true
I20240502 07:59:41.838471 20935 raft_consensus.cc:2454] T 04659b5e3504411987dafc49a5fa5ea7 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35642aa2099c40e59bc2092604078e07 in term 0.
I20240502 07:59:41.838572 20539 raft_consensus.cc:2454] T 04659b5e3504411987dafc49a5fa5ea7 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35642aa2099c40e59bc2092604078e07 in term 0.
I20240502 07:59:41.838912 20341 leader_election.cc:302] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 35642aa2099c40e59bc2092604078e07, b443e68b6212450286869124684bdfff, b5f471f8b5944986a7d878cf6ac550b8, e951f009258e473cb57cf8773c7975d4; no voters:
I20240502 07:59:41.839258 21441 raft_consensus.cc:2781] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:41.839324 21441 raft_consensus.cc:482] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:41.839365 21441 raft_consensus.cc:3037] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:41.839747 20275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04659b5e3504411987dafc49a5fa5ea7" candidate_uuid: "35642aa2099c40e59bc2092604078e07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" is_pre_election: true
I20240502 07:59:41.840432 20275 raft_consensus.cc:2454] T 04659b5e3504411987dafc49a5fa5ea7 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35642aa2099c40e59bc2092604078e07 in term 0.
I20240502 07:59:41.840440 20803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04659b5e3504411987dafc49a5fa5ea7" candidate_uuid: "35642aa2099c40e59bc2092604078e07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15e23a34059456788f9775031e27ec0" is_pre_election: true
I20240502 07:59:41.840582 21441 raft_consensus.cc:504] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } }
I20240502 07:59:41.840656 20803 raft_consensus.cc:2454] T 04659b5e3504411987dafc49a5fa5ea7 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35642aa2099c40e59bc2092604078e07 in term 0.
I20240502 07:59:41.841226 20935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04659b5e3504411987dafc49a5fa5ea7" candidate_uuid: "35642aa2099c40e59bc2092604078e07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f471f8b5944986a7d878cf6ac550b8"
I20240502 07:59:41.841329 20935 raft_consensus.cc:3037] T 04659b5e3504411987dafc49a5fa5ea7 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:41.841420 20671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04659b5e3504411987dafc49a5fa5ea7" candidate_uuid: "35642aa2099c40e59bc2092604078e07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b443e68b6212450286869124684bdfff"
I20240502 07:59:41.841532 20671 raft_consensus.cc:3037] T 04659b5e3504411987dafc49a5fa5ea7 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:41.841653 20275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04659b5e3504411987dafc49a5fa5ea7" candidate_uuid: "35642aa2099c40e59bc2092604078e07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7604c0eb3bf4a7cb260b564a94a1191"
I20240502 07:59:41.841753 20275 raft_consensus.cc:3037] T 04659b5e3504411987dafc49a5fa5ea7 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:41.842360 20935 raft_consensus.cc:2454] T 04659b5e3504411987dafc49a5fa5ea7 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35642aa2099c40e59bc2092604078e07 in term 1.
I20240502 07:59:41.842356 20803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04659b5e3504411987dafc49a5fa5ea7" candidate_uuid: "35642aa2099c40e59bc2092604078e07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15e23a34059456788f9775031e27ec0"
I20240502 07:59:41.842448 20803 raft_consensus.cc:3037] T 04659b5e3504411987dafc49a5fa5ea7 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:41.842579 20671 raft_consensus.cc:2454] T 04659b5e3504411987dafc49a5fa5ea7 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35642aa2099c40e59bc2092604078e07 in term 1.
I20240502 07:59:41.843533 21441 leader_election.cc:288] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [CANDIDATE]: Term 1 election: Requested vote from peers b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577), e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315), b443e68b6212450286869124684bdfff (127.3.139.69:37923), c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085), 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703), b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331)
I20240502 07:59:41.842694 20275 raft_consensus.cc:2454] T 04659b5e3504411987dafc49a5fa5ea7 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35642aa2099c40e59bc2092604078e07 in term 1.
I20240502 07:59:41.843832 20803 raft_consensus.cc:2454] T 04659b5e3504411987dafc49a5fa5ea7 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35642aa2099c40e59bc2092604078e07 in term 1.
I20240502 07:59:41.843920 20142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04659b5e3504411987dafc49a5fa5ea7" candidate_uuid: "35642aa2099c40e59bc2092604078e07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7180fe4574c09a8dce7c27500aab2"
I20240502 07:59:41.843917 20143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04659b5e3504411987dafc49a5fa5ea7" candidate_uuid: "35642aa2099c40e59bc2092604078e07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7180fe4574c09a8dce7c27500aab2" is_pre_election: true
I20240502 07:59:41.844096 20539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04659b5e3504411987dafc49a5fa5ea7" candidate_uuid: "35642aa2099c40e59bc2092604078e07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e951f009258e473cb57cf8773c7975d4"
I20240502 07:59:41.844111 20341 leader_election.cc:302] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 35642aa2099c40e59bc2092604078e07, b443e68b6212450286869124684bdfff, b5f471f8b5944986a7d878cf6ac550b8, c15e23a34059456788f9775031e27ec0; no voters:
I20240502 07:59:41.844141 20142 raft_consensus.cc:3037] T 04659b5e3504411987dafc49a5fa5ea7 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:41.844188 20539 raft_consensus.cc:3037] T 04659b5e3504411987dafc49a5fa5ea7 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:41.844463 21441 raft_consensus.cc:2781] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:41.844647 21441 raft_consensus.cc:686] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [term 1 LEADER]: Becoming Leader. State: Replica: 35642aa2099c40e59bc2092604078e07, State: Running, Role: LEADER
I20240502 07:59:41.844997 21441 consensus_queue.cc:237] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } }
I20240502 07:59:41.845207 20142 raft_consensus.cc:2454] T 04659b5e3504411987dafc49a5fa5ea7 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35642aa2099c40e59bc2092604078e07 in term 1.
I20240502 07:59:41.845496 20539 raft_consensus.cc:2454] T 04659b5e3504411987dafc49a5fa5ea7 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35642aa2099c40e59bc2092604078e07 in term 1.
I20240502 07:59:41.848268 20000 catalog_manager.cc:5484] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 reported cstate change: term changed from 0 to 1, leader changed from <none> to 35642aa2099c40e59bc2092604078e07 (127.3.139.67). New cstate: current_term: 1 leader_uuid: "35642aa2099c40e59bc2092604078e07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: UNKNOWN } } }
W20240502 07:59:41.852396 20586 tablet.cc:2369] T 04659b5e3504411987dafc49a5fa5ea7 P e951f009258e473cb57cf8773c7975d4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 07:59:41.854933 21432 raft_consensus.cc:482] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:41.855021 21432 raft_consensus.cc:504] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:41.856285 21432 leader_election.cc:288] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703), b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331), 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587), e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315), c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085), b443e68b6212450286869124684bdfff (127.3.139.69:37923)
I20240502 07:59:41.861209 20142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15bb9480e88b41219cc91168cc438c70" candidate_uuid: "b5f471f8b5944986a7d878cf6ac550b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7180fe4574c09a8dce7c27500aab2" is_pre_election: true
I20240502 07:59:41.861344 20142 raft_consensus.cc:2454] T 15bb9480e88b41219cc91168cc438c70 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5f471f8b5944986a7d878cf6ac550b8 in term 0.
I20240502 07:59:41.862126 20275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15bb9480e88b41219cc91168cc438c70" candidate_uuid: "b5f471f8b5944986a7d878cf6ac550b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" is_pre_election: true
I20240502 07:59:41.862264 20275 raft_consensus.cc:2454] T 15bb9480e88b41219cc91168cc438c70 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5f471f8b5944986a7d878cf6ac550b8 in term 0.
I20240502 07:59:41.862478 20671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15bb9480e88b41219cc91168cc438c70" candidate_uuid: "b5f471f8b5944986a7d878cf6ac550b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b443e68b6212450286869124684bdfff" is_pre_election: true
I20240502 07:59:41.862607 20671 raft_consensus.cc:2454] T 15bb9480e88b41219cc91168cc438c70 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5f471f8b5944986a7d878cf6ac550b8 in term 0.
I20240502 07:59:41.862874 20870 leader_election.cc:302] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 29a7180fe4574c09a8dce7c27500aab2, b443e68b6212450286869124684bdfff, b5f471f8b5944986a7d878cf6ac550b8, b7604c0eb3bf4a7cb260b564a94a1191; no voters:
I20240502 07:59:41.863168 21432 raft_consensus.cc:2781] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:41.863232 21432 raft_consensus.cc:482] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:41.863267 21432 raft_consensus.cc:3037] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:41.863914 20803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15bb9480e88b41219cc91168cc438c70" candidate_uuid: "b5f471f8b5944986a7d878cf6ac550b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15e23a34059456788f9775031e27ec0" is_pre_election: true
I20240502 07:59:41.864037 20803 raft_consensus.cc:2454] T 15bb9480e88b41219cc91168cc438c70 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5f471f8b5944986a7d878cf6ac550b8 in term 0.
I20240502 07:59:41.864269 21432 raft_consensus.cc:504] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:41.864339 20539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15bb9480e88b41219cc91168cc438c70" candidate_uuid: "b5f471f8b5944986a7d878cf6ac550b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e951f009258e473cb57cf8773c7975d4" is_pre_election: true
I20240502 07:59:41.864439 20539 raft_consensus.cc:2454] T 15bb9480e88b41219cc91168cc438c70 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5f471f8b5944986a7d878cf6ac550b8 in term 0.
I20240502 07:59:41.864825 21432 leader_election.cc:288] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [CANDIDATE]: Term 1 election: Requested vote from peers 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703), b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331), 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587), e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315), c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085), b443e68b6212450286869124684bdfff (127.3.139.69:37923)
I20240502 07:59:41.865077 20275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15bb9480e88b41219cc91168cc438c70" candidate_uuid: "b5f471f8b5944986a7d878cf6ac550b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7604c0eb3bf4a7cb260b564a94a1191"
I20240502 07:59:41.865135 20671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15bb9480e88b41219cc91168cc438c70" candidate_uuid: "b5f471f8b5944986a7d878cf6ac550b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b443e68b6212450286869124684bdfff"
I20240502 07:59:41.865197 20275 raft_consensus.cc:3037] T 15bb9480e88b41219cc91168cc438c70 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:41.865224 20671 raft_consensus.cc:3037] T 15bb9480e88b41219cc91168cc438c70 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:41.865306 20539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15bb9480e88b41219cc91168cc438c70" candidate_uuid: "b5f471f8b5944986a7d878cf6ac550b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e951f009258e473cb57cf8773c7975d4"
I20240502 07:59:41.865306 20142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15bb9480e88b41219cc91168cc438c70" candidate_uuid: "b5f471f8b5944986a7d878cf6ac550b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7180fe4574c09a8dce7c27500aab2"
I20240502 07:59:41.865406 20142 raft_consensus.cc:3037] T 15bb9480e88b41219cc91168cc438c70 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:41.865406 20539 raft_consensus.cc:3037] T 15bb9480e88b41219cc91168cc438c70 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:41.866056 20671 raft_consensus.cc:2454] T 15bb9480e88b41219cc91168cc438c70 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5f471f8b5944986a7d878cf6ac550b8 in term 1.
I20240502 07:59:41.866056 20275 raft_consensus.cc:2454] T 15bb9480e88b41219cc91168cc438c70 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5f471f8b5944986a7d878cf6ac550b8 in term 1.
I20240502 07:59:41.866273 20142 raft_consensus.cc:2454] T 15bb9480e88b41219cc91168cc438c70 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5f471f8b5944986a7d878cf6ac550b8 in term 1.
I20240502 07:59:41.866273 20539 raft_consensus.cc:2454] T 15bb9480e88b41219cc91168cc438c70 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5f471f8b5944986a7d878cf6ac550b8 in term 1.
I20240502 07:59:41.866410 20407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15bb9480e88b41219cc91168cc438c70" candidate_uuid: "b5f471f8b5944986a7d878cf6ac550b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35642aa2099c40e59bc2092604078e07" is_pre_election: true
I20240502 07:59:41.866492 20803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15bb9480e88b41219cc91168cc438c70" candidate_uuid: "b5f471f8b5944986a7d878cf6ac550b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15e23a34059456788f9775031e27ec0"
I20240502 07:59:41.866602 20803 raft_consensus.cc:3037] T 15bb9480e88b41219cc91168cc438c70 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:41.866605 20407 raft_consensus.cc:2454] T 15bb9480e88b41219cc91168cc438c70 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5f471f8b5944986a7d878cf6ac550b8 in term 0.
I20240502 07:59:41.866688 20406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15bb9480e88b41219cc91168cc438c70" candidate_uuid: "b5f471f8b5944986a7d878cf6ac550b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35642aa2099c40e59bc2092604078e07"
I20240502 07:59:41.866776 20406 raft_consensus.cc:3037] T 15bb9480e88b41219cc91168cc438c70 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:41.867147 20869 leader_election.cc:302] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 29a7180fe4574c09a8dce7c27500aab2, b443e68b6212450286869124684bdfff, b5f471f8b5944986a7d878cf6ac550b8, e951f009258e473cb57cf8773c7975d4; no voters:
I20240502 07:59:41.867383 21432 raft_consensus.cc:2781] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:41.867578 21432 raft_consensus.cc:686] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 LEADER]: Becoming Leader. State: Replica: b5f471f8b5944986a7d878cf6ac550b8, State: Running, Role: LEADER
I20240502 07:59:41.867633 20406 raft_consensus.cc:2454] T 15bb9480e88b41219cc91168cc438c70 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5f471f8b5944986a7d878cf6ac550b8 in term 1.
I20240502 07:59:41.867952 20803 raft_consensus.cc:2454] T 15bb9480e88b41219cc91168cc438c70 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5f471f8b5944986a7d878cf6ac550b8 in term 1.
I20240502 07:59:41.867935 21432 consensus_queue.cc:237] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
W20240502 07:59:41.868958 20982 tablet.cc:2369] T 04659b5e3504411987dafc49a5fa5ea7 P b5f471f8b5944986a7d878cf6ac550b8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 07:59:41.871404 20000 catalog_manager.cc:5484] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 reported cstate change: term changed from 0 to 1, leader changed from <none> to b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71). New cstate: current_term: 1 leader_uuid: "b5f471f8b5944986a7d878cf6ac550b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: HEALTHY } } }
I20240502 07:59:41.878572 21418 raft_consensus.cc:482] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:41.878684 21418 raft_consensus.cc:504] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:41.879942 21418 leader_election.cc:288] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703), b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331), 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587), b443e68b6212450286869124684bdfff (127.3.139.69:37923), c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085), b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577)
I20240502 07:59:41.884649 20275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a973de06330c487ba54a8a36b1e31b54" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" is_pre_election: true
I20240502 07:59:41.884805 20275 raft_consensus.cc:2454] T a973de06330c487ba54a8a36b1e31b54 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e951f009258e473cb57cf8773c7975d4 in term 0.
I20240502 07:59:41.885380 20142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a973de06330c487ba54a8a36b1e31b54" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7180fe4574c09a8dce7c27500aab2" is_pre_election: true
I20240502 07:59:41.885380 20803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a973de06330c487ba54a8a36b1e31b54" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15e23a34059456788f9775031e27ec0" is_pre_election: true
I20240502 07:59:41.885505 20803 raft_consensus.cc:2454] T a973de06330c487ba54a8a36b1e31b54 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e951f009258e473cb57cf8773c7975d4 in term 0.
I20240502 07:59:41.885505 20142 raft_consensus.cc:2454] T a973de06330c487ba54a8a36b1e31b54 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e951f009258e473cb57cf8773c7975d4 in term 0.
I20240502 07:59:41.885869 20473 leader_election.cc:302] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 29a7180fe4574c09a8dce7c27500aab2, b7604c0eb3bf4a7cb260b564a94a1191, c15e23a34059456788f9775031e27ec0, e951f009258e473cb57cf8773c7975d4; no voters:
I20240502 07:59:41.886015 20406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a973de06330c487ba54a8a36b1e31b54" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35642aa2099c40e59bc2092604078e07" is_pre_election: true
I20240502 07:59:41.886138 21418 raft_consensus.cc:2781] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:41.886142 20406 raft_consensus.cc:2454] T a973de06330c487ba54a8a36b1e31b54 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e951f009258e473cb57cf8773c7975d4 in term 0.
I20240502 07:59:41.886205 21418 raft_consensus.cc:482] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:41.886250 21418 raft_consensus.cc:3037] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:41.887495 21418 raft_consensus.cc:504] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:41.887677 20671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a973de06330c487ba54a8a36b1e31b54" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b443e68b6212450286869124684bdfff" is_pre_election: true
I20240502 07:59:41.887800 20671 raft_consensus.cc:2454] T a973de06330c487ba54a8a36b1e31b54 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e951f009258e473cb57cf8773c7975d4 in term 0.
I20240502 07:59:41.888067 21418 leader_election.cc:288] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [CANDIDATE]: Term 1 election: Requested vote from peers 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703), b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331), 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587), b443e68b6212450286869124684bdfff (127.3.139.69:37923), c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085), b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577)
I20240502 07:59:41.888442 20406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a973de06330c487ba54a8a36b1e31b54" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35642aa2099c40e59bc2092604078e07"
I20240502 07:59:41.888548 20406 raft_consensus.cc:3037] T a973de06330c487ba54a8a36b1e31b54 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:41.888530 20803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a973de06330c487ba54a8a36b1e31b54" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15e23a34059456788f9775031e27ec0"
I20240502 07:59:41.888625 20803 raft_consensus.cc:3037] T a973de06330c487ba54a8a36b1e31b54 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:41.888726 20142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a973de06330c487ba54a8a36b1e31b54" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7180fe4574c09a8dce7c27500aab2"
I20240502 07:59:41.888840 20142 raft_consensus.cc:3037] T a973de06330c487ba54a8a36b1e31b54 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:41.889431 20406 raft_consensus.cc:2454] T a973de06330c487ba54a8a36b1e31b54 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e951f009258e473cb57cf8773c7975d4 in term 1.
I20240502 07:59:41.889431 20803 raft_consensus.cc:2454] T a973de06330c487ba54a8a36b1e31b54 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e951f009258e473cb57cf8773c7975d4 in term 1.
I20240502 07:59:41.889428 20275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a973de06330c487ba54a8a36b1e31b54" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7604c0eb3bf4a7cb260b564a94a1191"
I20240502 07:59:41.889533 20275 raft_consensus.cc:3037] T a973de06330c487ba54a8a36b1e31b54 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:41.889784 20142 raft_consensus.cc:2454] T a973de06330c487ba54a8a36b1e31b54 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e951f009258e473cb57cf8773c7975d4 in term 1.
I20240502 07:59:41.889832 20671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a973de06330c487ba54a8a36b1e31b54" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b443e68b6212450286869124684bdfff"
I20240502 07:59:41.889937 20671 raft_consensus.cc:3037] T a973de06330c487ba54a8a36b1e31b54 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:41.890008 20935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a973de06330c487ba54a8a36b1e31b54" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f471f8b5944986a7d878cf6ac550b8" is_pre_election: true
I20240502 07:59:41.890125 20935 raft_consensus.cc:2454] T a973de06330c487ba54a8a36b1e31b54 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e951f009258e473cb57cf8773c7975d4 in term 0.
I20240502 07:59:41.890165 20473 leader_election.cc:302] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 29a7180fe4574c09a8dce7c27500aab2, 35642aa2099c40e59bc2092604078e07, c15e23a34059456788f9775031e27ec0, e951f009258e473cb57cf8773c7975d4; no voters:
I20240502 07:59:41.890326 20934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a973de06330c487ba54a8a36b1e31b54" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f471f8b5944986a7d878cf6ac550b8"
I20240502 07:59:41.890408 20934 raft_consensus.cc:3037] T a973de06330c487ba54a8a36b1e31b54 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:41.890398 21418 raft_consensus.cc:2781] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:41.890486 20275 raft_consensus.cc:2454] T a973de06330c487ba54a8a36b1e31b54 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e951f009258e473cb57cf8773c7975d4 in term 1.
I20240502 07:59:41.890579 21418 raft_consensus.cc:686] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [term 1 LEADER]: Becoming Leader. State: Replica: e951f009258e473cb57cf8773c7975d4, State: Running, Role: LEADER
I20240502 07:59:41.890796 20671 raft_consensus.cc:2454] T a973de06330c487ba54a8a36b1e31b54 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e951f009258e473cb57cf8773c7975d4 in term 1.
I20240502 07:59:41.890923 21418 consensus_queue.cc:237] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:41.891347 20934 raft_consensus.cc:2454] T a973de06330c487ba54a8a36b1e31b54 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e951f009258e473cb57cf8773c7975d4 in term 1.
I20240502 07:59:41.894361 20000 catalog_manager.cc:5484] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to e951f009258e473cb57cf8773c7975d4 (127.3.139.68). New cstate: current_term: 1 leader_uuid: "e951f009258e473cb57cf8773c7975d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: UNKNOWN } } }
W20240502 07:59:41.920897 20454 tablet.cc:2369] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240502 07:59:41.946583 20850 tablet.cc:2369] T 04659b5e3504411987dafc49a5fa5ea7 P c15e23a34059456788f9775031e27ec0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240502 07:59:42.003084 20322 tablet.cc:2369] T 04659b5e3504411987dafc49a5fa5ea7 P b7604c0eb3bf4a7cb260b564a94a1191: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240502 07:59:42.009253 21466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:42.009506 21466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:42.013253 21466 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_0'
I20240502 07:59:42.096591 20934 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:42.097430 20803 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:42.097538 20671 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:42.097726 20406 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:42.098379 21464 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:42.098783 21464 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:42.099004 21464 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:42.104113 20275 raft_consensus.cc:1261] T 04659b5e3504411987dafc49a5fa5ea7 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:42.105068 20934 raft_consensus.cc:1261] T 04659b5e3504411987dafc49a5fa5ea7 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:42.105373 20671 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:42.105453 21441 consensus_queue.cc:1035] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:42.105911 20142 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:42.106537 20406 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:42.106988 20802 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:42.107578 20802 raft_consensus.cc:1261] T 04659b5e3504411987dafc49a5fa5ea7 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:42.108016 20142 raft_consensus.cc:1261] T 04659b5e3504411987dafc49a5fa5ea7 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:42.108022 21456 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:42.108134 20539 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:42.108472 21456 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:42.108549 20671 raft_consensus.cc:1261] T 04659b5e3504411987dafc49a5fa5ea7 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:42.108772 21456 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:42.109051 21456 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:42.109369 21439 consensus_queue.cc:1035] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:42.109387 21418 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:42.109689 21439 consensus_queue.cc:1035] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:42.109979 21439 consensus_queue.cc:1035] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:42.110004 20275 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:42.110225 21439 consensus_queue.cc:1035] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:42.114035 20275 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:42.114081 20143 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:42.114437 21432 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:42.114786 21432 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:42.115027 20539 raft_consensus.cc:1261] T 04659b5e3504411987dafc49a5fa5ea7 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:42.117007 21418 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:42.119421 21439 consensus_queue.cc:1035] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:42.120390 21464 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:42.140375 21510 mvcc.cc:205] Tried to move back new op lower bound from 7023152259461390336 to 7023152258629079040. Current Snapshot: MvccSnapshot[applied={T|T < 7023152259461390336}]
I20240502 07:59:42.144778 21530 mvcc.cc:205] Tried to move back new op lower bound from 7023152259461390336 to 7023152258629079040. Current Snapshot: MvccSnapshot[applied={T|T < 7023152259461390336}]
I20240502 07:59:42.158579 21513 mvcc.cc:205] Tried to move back new op lower bound from 7023152259500883968 to 7023152258441641984. Current Snapshot: MvccSnapshot[applied={T|T < 7023152259476799488}]
I20240502 07:59:42.160142 21523 mvcc.cc:205] Tried to move back new op lower bound from 7023152259623182336 to 7023152258629079040. Current Snapshot: MvccSnapshot[applied={T|T < 7023152259461390336}]
I20240502 07:59:42.163740 21500 mvcc.cc:205] Tried to move back new op lower bound from 7023152259500883968 to 7023152258441641984. Current Snapshot: MvccSnapshot[applied={T|T < 7023152259476799488}]
I20240502 07:59:42.166133 21505 mvcc.cc:205] Tried to move back new op lower bound from 7023152259461390336 to 7023152258629079040. Current Snapshot: MvccSnapshot[applied={T|T < 7023152259461390336}]
I20240502 07:59:42.179977 21537 mvcc.cc:205] Tried to move back new op lower bound from 7023152259461390336 to 7023152258629079040. Current Snapshot: MvccSnapshot[applied={T|T < 7023152259461390336}]
INSERT report
rows total: 2000
time total: 191.557 ms
time per row: 0.0957784 ms
I20240502 07:59:42.254283 19997 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33518:
name: "rebalance_test_table_1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 7
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240502 07:59:42.254863 19997 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_1 in case of a server failure: 8 tablet servers would be needed, 7 are available. Consider bringing up more tablet servers.
I20240502 07:59:42.259811 20122 tablet_service.cc:1467] Processing CreateTablet for tablet 1453e833d0ad494583c44b5496088f6d (DEFAULT_TABLE table=rebalance_test_table_1 [id=dd634821c68c405c9e104a7053619a61]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:42.259848 20253 tablet_service.cc:1467] Processing CreateTablet for tablet 1453e833d0ad494583c44b5496088f6d (DEFAULT_TABLE table=rebalance_test_table_1 [id=dd634821c68c405c9e104a7053619a61]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:42.260128 20253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1453e833d0ad494583c44b5496088f6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:42.260130 20122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1453e833d0ad494583c44b5496088f6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:42.260977 20649 tablet_service.cc:1467] Processing CreateTablet for tablet 4ce741ca5e264ccfbdde163add361886 (DEFAULT_TABLE table=rebalance_test_table_1 [id=dd634821c68c405c9e104a7053619a61]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:42.261243 20649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ce741ca5e264ccfbdde163add361886. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:42.262154 21408 tablet_bootstrap.cc:492] T 1453e833d0ad494583c44b5496088f6d P 29a7180fe4574c09a8dce7c27500aab2: Bootstrap starting.
I20240502 07:59:42.262356 20651 tablet_service.cc:1467] Processing CreateTablet for tablet 1453e833d0ad494583c44b5496088f6d (DEFAULT_TABLE table=rebalance_test_table_1 [id=dd634821c68c405c9e104a7053619a61]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:42.262545 20651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1453e833d0ad494583c44b5496088f6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:42.262684 20914 tablet_service.cc:1467] Processing CreateTablet for tablet 1453e833d0ad494583c44b5496088f6d (DEFAULT_TABLE table=rebalance_test_table_1 [id=dd634821c68c405c9e104a7053619a61]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:42.262933 20914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1453e833d0ad494583c44b5496088f6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:42.262945 21408 tablet_bootstrap.cc:654] T 1453e833d0ad494583c44b5496088f6d P 29a7180fe4574c09a8dce7c27500aab2: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:42.263900 20649 tablet_service.cc:1467] Processing CreateTablet for tablet 1bc5937510214c3da7731d3ad2a67842 (DEFAULT_TABLE table=rebalance_test_table_1 [id=dd634821c68c405c9e104a7053619a61]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:42.264093 20649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bc5937510214c3da7731d3ad2a67842. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:42.264658 21406 tablet_bootstrap.cc:492] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8: Bootstrap starting.
I20240502 07:59:42.264703 21413 tablet_bootstrap.cc:492] T 1453e833d0ad494583c44b5496088f6d P b7604c0eb3bf4a7cb260b564a94a1191: Bootstrap starting.
I20240502 07:59:42.264719 20385 tablet_service.cc:1467] Processing CreateTablet for tablet 1453e833d0ad494583c44b5496088f6d (DEFAULT_TABLE table=rebalance_test_table_1 [id=dd634821c68c405c9e104a7053619a61]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:42.264830 20915 tablet_service.cc:1467] Processing CreateTablet for tablet 4ce741ca5e264ccfbdde163add361886 (DEFAULT_TABLE table=rebalance_test_table_1 [id=dd634821c68c405c9e104a7053619a61]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:42.264952 20385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1453e833d0ad494583c44b5496088f6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:42.265025 20915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ce741ca5e264ccfbdde163add361886. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:42.265424 21406 tablet_bootstrap.cc:654] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:42.265460 21413 tablet_bootstrap.cc:654] T 1453e833d0ad494583c44b5496088f6d P b7604c0eb3bf4a7cb260b564a94a1191: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:42.265951 21407 tablet_bootstrap.cc:492] T 4ce741ca5e264ccfbdde163add361886 P b443e68b6212450286869124684bdfff: Bootstrap starting.
I20240502 07:59:42.266678 21407 tablet_bootstrap.cc:654] T 4ce741ca5e264ccfbdde163add361886 P b443e68b6212450286869124684bdfff: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:42.266779 20913 tablet_service.cc:1467] Processing CreateTablet for tablet 1bc5937510214c3da7731d3ad2a67842 (DEFAULT_TABLE table=rebalance_test_table_1 [id=dd634821c68c405c9e104a7053619a61]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:42.266897 21422 tablet_bootstrap.cc:492] T 1453e833d0ad494583c44b5496088f6d P 35642aa2099c40e59bc2092604078e07: Bootstrap starting.
I20240502 07:59:42.266980 20913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bc5937510214c3da7731d3ad2a67842. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:42.267102 20387 tablet_service.cc:1467] Processing CreateTablet for tablet 4ce741ca5e264ccfbdde163add361886 (DEFAULT_TABLE table=rebalance_test_table_1 [id=dd634821c68c405c9e104a7053619a61]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:42.267316 20387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ce741ca5e264ccfbdde163add361886. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:42.267392 20386 tablet_service.cc:1467] Processing CreateTablet for tablet 1bc5937510214c3da7731d3ad2a67842 (DEFAULT_TABLE table=rebalance_test_table_1 [id=dd634821c68c405c9e104a7053619a61]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:42.267587 20386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bc5937510214c3da7731d3ad2a67842. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:42.267771 21422 tablet_bootstrap.cc:654] T 1453e833d0ad494583c44b5496088f6d P 35642aa2099c40e59bc2092604078e07: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:42.269734 21422 tablet_bootstrap.cc:492] T 1453e833d0ad494583c44b5496088f6d P 35642aa2099c40e59bc2092604078e07: No bootstrap required, opened a new log
I20240502 07:59:42.269815 21422 ts_tablet_manager.cc:1367] T 1453e833d0ad494583c44b5496088f6d P 35642aa2099c40e59bc2092604078e07: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240502 07:59:42.269852 20518 tablet_service.cc:1467] Processing CreateTablet for tablet 4ce741ca5e264ccfbdde163add361886 (DEFAULT_TABLE table=rebalance_test_table_1 [id=dd634821c68c405c9e104a7053619a61]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:42.270100 20518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ce741ca5e264ccfbdde163add361886. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:42.270184 21422 raft_consensus.cc:348] T 1453e833d0ad494583c44b5496088f6d P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.270291 21422 raft_consensus.cc:374] T 1453e833d0ad494583c44b5496088f6d P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:42.270321 21422 raft_consensus.cc:729] T 1453e833d0ad494583c44b5496088f6d P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35642aa2099c40e59bc2092604078e07, State: Initialized, Role: FOLLOWER
I20240502 07:59:42.270355 20123 tablet_service.cc:1467] Processing CreateTablet for tablet 1bc5937510214c3da7731d3ad2a67842 (DEFAULT_TABLE table=rebalance_test_table_1 [id=dd634821c68c405c9e104a7053619a61]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:42.270355 20782 tablet_service.cc:1467] Processing CreateTablet for tablet 1453e833d0ad494583c44b5496088f6d (DEFAULT_TABLE table=rebalance_test_table_1 [id=dd634821c68c405c9e104a7053619a61]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:42.270442 21422 consensus_queue.cc:260] T 1453e833d0ad494583c44b5496088f6d P 35642aa2099c40e59bc2092604078e07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.270586 20782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1453e833d0ad494583c44b5496088f6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:42.270586 20123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bc5937510214c3da7731d3ad2a67842. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:42.270694 21422 ts_tablet_manager.cc:1398] T 1453e833d0ad494583c44b5496088f6d P 35642aa2099c40e59bc2092604078e07: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:42.270841 21422 tablet_bootstrap.cc:492] T 4ce741ca5e264ccfbdde163add361886 P 35642aa2099c40e59bc2092604078e07: Bootstrap starting.
I20240502 07:59:42.271307 20517 tablet_service.cc:1467] Processing CreateTablet for tablet 1bc5937510214c3da7731d3ad2a67842 (DEFAULT_TABLE table=rebalance_test_table_1 [id=dd634821c68c405c9e104a7053619a61]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:42.271538 20517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bc5937510214c3da7731d3ad2a67842. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:42.271590 21422 tablet_bootstrap.cc:654] T 4ce741ca5e264ccfbdde163add361886 P 35642aa2099c40e59bc2092604078e07: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:42.272236 20519 tablet_service.cc:1467] Processing CreateTablet for tablet 1453e833d0ad494583c44b5496088f6d (DEFAULT_TABLE table=rebalance_test_table_1 [id=dd634821c68c405c9e104a7053619a61]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:42.272436 20519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1453e833d0ad494583c44b5496088f6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:42.272570 20781 tablet_service.cc:1467] Processing CreateTablet for tablet 4ce741ca5e264ccfbdde163add361886 (DEFAULT_TABLE table=rebalance_test_table_1 [id=dd634821c68c405c9e104a7053619a61]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:42.272794 20781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ce741ca5e264ccfbdde163add361886. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:42.274299 21404 tablet_bootstrap.cc:492] T 1bc5937510214c3da7731d3ad2a67842 P e951f009258e473cb57cf8773c7975d4: Bootstrap starting.
I20240502 07:59:42.274507 20783 tablet_service.cc:1467] Processing CreateTablet for tablet 1bc5937510214c3da7731d3ad2a67842 (DEFAULT_TABLE table=rebalance_test_table_1 [id=dd634821c68c405c9e104a7053619a61]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:42.274726 20783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bc5937510214c3da7731d3ad2a67842. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:42.275167 21404 tablet_bootstrap.cc:654] T 1bc5937510214c3da7731d3ad2a67842 P e951f009258e473cb57cf8773c7975d4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:42.275324 21405 tablet_bootstrap.cc:492] T 1453e833d0ad494583c44b5496088f6d P c15e23a34059456788f9775031e27ec0: Bootstrap starting.
I20240502 07:59:42.276070 21422 tablet_bootstrap.cc:492] T 4ce741ca5e264ccfbdde163add361886 P 35642aa2099c40e59bc2092604078e07: No bootstrap required, opened a new log
I20240502 07:59:42.276141 21422 ts_tablet_manager.cc:1367] T 4ce741ca5e264ccfbdde163add361886 P 35642aa2099c40e59bc2092604078e07: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 07:59:42.276156 21405 tablet_bootstrap.cc:654] T 1453e833d0ad494583c44b5496088f6d P c15e23a34059456788f9775031e27ec0: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:42.276248 21406 tablet_bootstrap.cc:492] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8: No bootstrap required, opened a new log
I20240502 07:59:42.276324 21406 ts_tablet_manager.cc:1367] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20240502 07:59:42.276549 21422 raft_consensus.cc:348] T 4ce741ca5e264ccfbdde163add361886 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.276656 21422 raft_consensus.cc:374] T 4ce741ca5e264ccfbdde163add361886 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:42.276688 21422 raft_consensus.cc:729] T 4ce741ca5e264ccfbdde163add361886 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35642aa2099c40e59bc2092604078e07, State: Initialized, Role: FOLLOWER
I20240502 07:59:42.276727 21406 raft_consensus.cc:348] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.276826 21406 raft_consensus.cc:374] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:42.276858 21406 raft_consensus.cc:729] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5f471f8b5944986a7d878cf6ac550b8, State: Initialized, Role: FOLLOWER
I20240502 07:59:42.276868 21404 tablet_bootstrap.cc:492] T 1bc5937510214c3da7731d3ad2a67842 P e951f009258e473cb57cf8773c7975d4: No bootstrap required, opened a new log
I20240502 07:59:42.276845 21422 consensus_queue.cc:260] T 4ce741ca5e264ccfbdde163add361886 P 35642aa2099c40e59bc2092604078e07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.276937 21404 ts_tablet_manager.cc:1367] T 1bc5937510214c3da7731d3ad2a67842 P e951f009258e473cb57cf8773c7975d4: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240502 07:59:42.276969 21406 consensus_queue.cc:260] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.277199 21422 ts_tablet_manager.cc:1398] T 4ce741ca5e264ccfbdde163add361886 P 35642aa2099c40e59bc2092604078e07: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:42.277213 21406 ts_tablet_manager.cc:1398] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:42.277221 20122 tablet_service.cc:1467] Processing CreateTablet for tablet 4ce741ca5e264ccfbdde163add361886 (DEFAULT_TABLE table=rebalance_test_table_1 [id=dd634821c68c405c9e104a7053619a61]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:42.277350 21422 tablet_bootstrap.cc:492] T 1bc5937510214c3da7731d3ad2a67842 P 35642aa2099c40e59bc2092604078e07: Bootstrap starting.
I20240502 07:59:42.277350 21406 tablet_bootstrap.cc:492] T 4ce741ca5e264ccfbdde163add361886 P b5f471f8b5944986a7d878cf6ac550b8: Bootstrap starting.
I20240502 07:59:42.277324 21404 raft_consensus.cc:348] T 1bc5937510214c3da7731d3ad2a67842 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:42.277451 20122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ce741ca5e264ccfbdde163add361886. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:42.277449 21404 raft_consensus.cc:374] T 1bc5937510214c3da7731d3ad2a67842 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:42.277483 21404 raft_consensus.cc:729] T 1bc5937510214c3da7731d3ad2a67842 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e951f009258e473cb57cf8773c7975d4, State: Initialized, Role: FOLLOWER
I20240502 07:59:42.277590 21404 consensus_queue.cc:260] T 1bc5937510214c3da7731d3ad2a67842 P e951f009258e473cb57cf8773c7975d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:42.277878 21404 ts_tablet_manager.cc:1398] T 1bc5937510214c3da7731d3ad2a67842 P e951f009258e473cb57cf8773c7975d4: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:42.278028 21404 tablet_bootstrap.cc:492] T 1453e833d0ad494583c44b5496088f6d P e951f009258e473cb57cf8773c7975d4: Bootstrap starting.
I20240502 07:59:42.278111 21406 tablet_bootstrap.cc:654] T 4ce741ca5e264ccfbdde163add361886 P b5f471f8b5944986a7d878cf6ac550b8: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:42.278111 21422 tablet_bootstrap.cc:654] T 1bc5937510214c3da7731d3ad2a67842 P 35642aa2099c40e59bc2092604078e07: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:42.278805 21404 tablet_bootstrap.cc:654] T 1453e833d0ad494583c44b5496088f6d P e951f009258e473cb57cf8773c7975d4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:42.279587 21406 tablet_bootstrap.cc:492] T 4ce741ca5e264ccfbdde163add361886 P b5f471f8b5944986a7d878cf6ac550b8: No bootstrap required, opened a new log
I20240502 07:59:42.279690 21406 ts_tablet_manager.cc:1367] T 4ce741ca5e264ccfbdde163add361886 P b5f471f8b5944986a7d878cf6ac550b8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:42.279769 21422 tablet_bootstrap.cc:492] T 1bc5937510214c3da7731d3ad2a67842 P 35642aa2099c40e59bc2092604078e07: No bootstrap required, opened a new log
I20240502 07:59:42.279834 21422 ts_tablet_manager.cc:1367] T 1bc5937510214c3da7731d3ad2a67842 P 35642aa2099c40e59bc2092604078e07: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:59:42.280073 20253 tablet_service.cc:1467] Processing CreateTablet for tablet 4ce741ca5e264ccfbdde163add361886 (DEFAULT_TABLE table=rebalance_test_table_1 [id=dd634821c68c405c9e104a7053619a61]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:42.280066 21406 raft_consensus.cc:348] T 4ce741ca5e264ccfbdde163add361886 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.280167 21406 raft_consensus.cc:374] T 4ce741ca5e264ccfbdde163add361886 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:42.280200 21406 raft_consensus.cc:729] T 4ce741ca5e264ccfbdde163add361886 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5f471f8b5944986a7d878cf6ac550b8, State: Initialized, Role: FOLLOWER
I20240502 07:59:42.280195 21422 raft_consensus.cc:348] T 1bc5937510214c3da7731d3ad2a67842 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:42.280287 21422 raft_consensus.cc:374] T 1bc5937510214c3da7731d3ad2a67842 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:42.280314 21422 raft_consensus.cc:729] T 1bc5937510214c3da7731d3ad2a67842 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35642aa2099c40e59bc2092604078e07, State: Initialized, Role: FOLLOWER
I20240502 07:59:42.280326 20253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ce741ca5e264ccfbdde163add361886. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:42.280313 21406 consensus_queue.cc:260] T 4ce741ca5e264ccfbdde163add361886 P b5f471f8b5944986a7d878cf6ac550b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.280431 21422 consensus_queue.cc:260] T 1bc5937510214c3da7731d3ad2a67842 P 35642aa2099c40e59bc2092604078e07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:42.280625 21406 ts_tablet_manager.cc:1398] T 4ce741ca5e264ccfbdde163add361886 P b5f471f8b5944986a7d878cf6ac550b8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:42.280666 21422 ts_tablet_manager.cc:1398] T 1bc5937510214c3da7731d3ad2a67842 P 35642aa2099c40e59bc2092604078e07: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:42.280784 21406 tablet_bootstrap.cc:492] T 1bc5937510214c3da7731d3ad2a67842 P b5f471f8b5944986a7d878cf6ac550b8: Bootstrap starting.
I20240502 07:59:42.281386 21406 tablet_bootstrap.cc:654] T 1bc5937510214c3da7731d3ad2a67842 P b5f471f8b5944986a7d878cf6ac550b8: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:42.282233 21406 tablet_bootstrap.cc:492] T 1bc5937510214c3da7731d3ad2a67842 P b5f471f8b5944986a7d878cf6ac550b8: No bootstrap required, opened a new log
I20240502 07:59:42.282300 21406 ts_tablet_manager.cc:1367] T 1bc5937510214c3da7731d3ad2a67842 P b5f471f8b5944986a7d878cf6ac550b8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:42.282408 20254 tablet_service.cc:1467] Processing CreateTablet for tablet 1bc5937510214c3da7731d3ad2a67842 (DEFAULT_TABLE table=rebalance_test_table_1 [id=dd634821c68c405c9e104a7053619a61]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:42.282619 20254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bc5937510214c3da7731d3ad2a67842. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:42.282671 21406 raft_consensus.cc:348] T 1bc5937510214c3da7731d3ad2a67842 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:42.282775 21406 raft_consensus.cc:374] T 1bc5937510214c3da7731d3ad2a67842 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:42.282809 21406 raft_consensus.cc:729] T 1bc5937510214c3da7731d3ad2a67842 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5f471f8b5944986a7d878cf6ac550b8, State: Initialized, Role: FOLLOWER
I20240502 07:59:42.282936 21406 consensus_queue.cc:260] T 1bc5937510214c3da7731d3ad2a67842 P b5f471f8b5944986a7d878cf6ac550b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:42.283232 21406 ts_tablet_manager.cc:1398] T 1bc5937510214c3da7731d3ad2a67842 P b5f471f8b5944986a7d878cf6ac550b8: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:42.286351 21413 tablet_bootstrap.cc:492] T 1453e833d0ad494583c44b5496088f6d P b7604c0eb3bf4a7cb260b564a94a1191: No bootstrap required, opened a new log
I20240502 07:59:42.286420 21404 tablet_bootstrap.cc:492] T 1453e833d0ad494583c44b5496088f6d P e951f009258e473cb57cf8773c7975d4: No bootstrap required, opened a new log
I20240502 07:59:42.286437 21413 ts_tablet_manager.cc:1367] T 1453e833d0ad494583c44b5496088f6d P b7604c0eb3bf4a7cb260b564a94a1191: Time spent bootstrapping tablet: real 0.022s user 0.001s sys 0.000s
I20240502 07:59:42.286515 21404 ts_tablet_manager.cc:1367] T 1453e833d0ad494583c44b5496088f6d P e951f009258e473cb57cf8773c7975d4: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 07:59:42.286809 21413 raft_consensus.cc:348] T 1453e833d0ad494583c44b5496088f6d P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.286901 21413 raft_consensus.cc:374] T 1453e833d0ad494583c44b5496088f6d P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:42.286936 21413 raft_consensus.cc:729] T 1453e833d0ad494583c44b5496088f6d P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7604c0eb3bf4a7cb260b564a94a1191, State: Initialized, Role: FOLLOWER
I20240502 07:59:42.286897 21404 raft_consensus.cc:348] T 1453e833d0ad494583c44b5496088f6d P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.287011 21404 raft_consensus.cc:374] T 1453e833d0ad494583c44b5496088f6d P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:42.287077 21404 raft_consensus.cc:729] T 1453e833d0ad494583c44b5496088f6d P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e951f009258e473cb57cf8773c7975d4, State: Initialized, Role: FOLLOWER
I20240502 07:59:42.287042 21413 consensus_queue.cc:260] T 1453e833d0ad494583c44b5496088f6d P b7604c0eb3bf4a7cb260b564a94a1191 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.287199 21404 consensus_queue.cc:260] T 1453e833d0ad494583c44b5496088f6d P e951f009258e473cb57cf8773c7975d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.287346 21413 ts_tablet_manager.cc:1398] T 1453e833d0ad494583c44b5496088f6d P b7604c0eb3bf4a7cb260b564a94a1191: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:42.287468 21404 ts_tablet_manager.cc:1398] T 1453e833d0ad494583c44b5496088f6d P e951f009258e473cb57cf8773c7975d4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:42.287510 21413 tablet_bootstrap.cc:492] T 4ce741ca5e264ccfbdde163add361886 P b7604c0eb3bf4a7cb260b564a94a1191: Bootstrap starting.
I20240502 07:59:42.287627 21404 tablet_bootstrap.cc:492] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4: Bootstrap starting.
I20240502 07:59:42.288311 21413 tablet_bootstrap.cc:654] T 4ce741ca5e264ccfbdde163add361886 P b7604c0eb3bf4a7cb260b564a94a1191: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:42.288352 21404 tablet_bootstrap.cc:654] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:42.289673 21407 tablet_bootstrap.cc:492] T 4ce741ca5e264ccfbdde163add361886 P b443e68b6212450286869124684bdfff: No bootstrap required, opened a new log
I20240502 07:59:42.289673 21405 tablet_bootstrap.cc:492] T 1453e833d0ad494583c44b5496088f6d P c15e23a34059456788f9775031e27ec0: No bootstrap required, opened a new log
I20240502 07:59:42.289770 21407 ts_tablet_manager.cc:1367] T 4ce741ca5e264ccfbdde163add361886 P b443e68b6212450286869124684bdfff: Time spent bootstrapping tablet: real 0.024s user 0.001s sys 0.000s
I20240502 07:59:42.289770 21405 ts_tablet_manager.cc:1367] T 1453e833d0ad494583c44b5496088f6d P c15e23a34059456788f9775031e27ec0: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240502 07:59:42.290206 21407 raft_consensus.cc:348] T 4ce741ca5e264ccfbdde163add361886 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.290324 21407 raft_consensus.cc:374] T 4ce741ca5e264ccfbdde163add361886 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:42.290356 21407 raft_consensus.cc:729] T 4ce741ca5e264ccfbdde163add361886 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b443e68b6212450286869124684bdfff, State: Initialized, Role: FOLLOWER
I20240502 07:59:42.290336 21405 raft_consensus.cc:348] T 1453e833d0ad494583c44b5496088f6d P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.290473 21405 raft_consensus.cc:374] T 1453e833d0ad494583c44b5496088f6d P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:42.290506 21405 raft_consensus.cc:729] T 1453e833d0ad494583c44b5496088f6d P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c15e23a34059456788f9775031e27ec0, State: Initialized, Role: FOLLOWER
I20240502 07:59:42.290465 21407 consensus_queue.cc:260] T 4ce741ca5e264ccfbdde163add361886 P b443e68b6212450286869124684bdfff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.290616 21405 consensus_queue.cc:260] T 1453e833d0ad494583c44b5496088f6d P c15e23a34059456788f9775031e27ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.290724 21407 ts_tablet_manager.cc:1398] T 4ce741ca5e264ccfbdde163add361886 P b443e68b6212450286869124684bdfff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:42.290882 21407 tablet_bootstrap.cc:492] T 1453e833d0ad494583c44b5496088f6d P b443e68b6212450286869124684bdfff: Bootstrap starting.
I20240502 07:59:42.290882 21405 ts_tablet_manager.cc:1398] T 1453e833d0ad494583c44b5496088f6d P c15e23a34059456788f9775031e27ec0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:42.291074 21405 tablet_bootstrap.cc:492] T 4ce741ca5e264ccfbdde163add361886 P c15e23a34059456788f9775031e27ec0: Bootstrap starting.
I20240502 07:59:42.291239 21404 tablet_bootstrap.cc:492] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4: No bootstrap required, opened a new log
I20240502 07:59:42.291304 21404 ts_tablet_manager.cc:1367] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:59:42.291709 21407 tablet_bootstrap.cc:654] T 1453e833d0ad494583c44b5496088f6d P b443e68b6212450286869124684bdfff: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:42.291687 21404 raft_consensus.cc:348] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.291795 21404 raft_consensus.cc:374] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:42.291824 21404 raft_consensus.cc:729] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e951f009258e473cb57cf8773c7975d4, State: Initialized, Role: FOLLOWER
I20240502 07:59:42.291899 21405 tablet_bootstrap.cc:654] T 4ce741ca5e264ccfbdde163add361886 P c15e23a34059456788f9775031e27ec0: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:42.291935 21404 consensus_queue.cc:260] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.292199 21404 ts_tablet_manager.cc:1398] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:42.293413 21413 tablet_bootstrap.cc:492] T 4ce741ca5e264ccfbdde163add361886 P b7604c0eb3bf4a7cb260b564a94a1191: No bootstrap required, opened a new log
I20240502 07:59:42.293490 21413 ts_tablet_manager.cc:1367] T 4ce741ca5e264ccfbdde163add361886 P b7604c0eb3bf4a7cb260b564a94a1191: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 07:59:42.293576 21407 tablet_bootstrap.cc:492] T 1453e833d0ad494583c44b5496088f6d P b443e68b6212450286869124684bdfff: No bootstrap required, opened a new log
I20240502 07:59:42.293644 21407 ts_tablet_manager.cc:1367] T 1453e833d0ad494583c44b5496088f6d P b443e68b6212450286869124684bdfff: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:59:42.293880 21413 raft_consensus.cc:348] T 4ce741ca5e264ccfbdde163add361886 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.293979 21413 raft_consensus.cc:374] T 4ce741ca5e264ccfbdde163add361886 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:42.294009 21413 raft_consensus.cc:729] T 4ce741ca5e264ccfbdde163add361886 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7604c0eb3bf4a7cb260b564a94a1191, State: Initialized, Role: FOLLOWER
I20240502 07:59:42.294121 21413 consensus_queue.cc:260] T 4ce741ca5e264ccfbdde163add361886 P b7604c0eb3bf4a7cb260b564a94a1191 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.294166 21407 raft_consensus.cc:348] T 1453e833d0ad494583c44b5496088f6d P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.294278 21407 raft_consensus.cc:374] T 1453e833d0ad494583c44b5496088f6d P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:42.294307 21407 raft_consensus.cc:729] T 1453e833d0ad494583c44b5496088f6d P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b443e68b6212450286869124684bdfff, State: Initialized, Role: FOLLOWER
I20240502 07:59:42.294397 21413 ts_tablet_manager.cc:1398] T 4ce741ca5e264ccfbdde163add361886 P b7604c0eb3bf4a7cb260b564a94a1191: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:42.294410 21407 consensus_queue.cc:260] T 1453e833d0ad494583c44b5496088f6d P b443e68b6212450286869124684bdfff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.294654 21407 ts_tablet_manager.cc:1398] T 1453e833d0ad494583c44b5496088f6d P b443e68b6212450286869124684bdfff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:42.294746 21413 tablet_bootstrap.cc:492] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191: Bootstrap starting.
I20240502 07:59:42.294811 21407 tablet_bootstrap.cc:492] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff: Bootstrap starting.
I20240502 07:59:42.295629 21407 tablet_bootstrap.cc:654] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:42.296139 21413 tablet_bootstrap.cc:654] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:42.296665 21407 tablet_bootstrap.cc:492] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff: No bootstrap required, opened a new log
I20240502 07:59:42.296808 21407 ts_tablet_manager.cc:1367] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:42.297089 21413 tablet_bootstrap.cc:492] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191: No bootstrap required, opened a new log
I20240502 07:59:42.297180 21413 ts_tablet_manager.cc:1367] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:42.297339 21407 raft_consensus.cc:348] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:42.297493 21407 raft_consensus.cc:374] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:42.297516 21405 tablet_bootstrap.cc:492] T 4ce741ca5e264ccfbdde163add361886 P c15e23a34059456788f9775031e27ec0: No bootstrap required, opened a new log
I20240502 07:59:42.297577 21407 raft_consensus.cc:729] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b443e68b6212450286869124684bdfff, State: Initialized, Role: FOLLOWER
I20240502 07:59:42.297588 21405 ts_tablet_manager.cc:1367] T 4ce741ca5e264ccfbdde163add361886 P c15e23a34059456788f9775031e27ec0: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 07:59:42.297570 21413 raft_consensus.cc:348] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:42.297678 21413 raft_consensus.cc:374] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:42.297711 21413 raft_consensus.cc:729] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7604c0eb3bf4a7cb260b564a94a1191, State: Initialized, Role: FOLLOWER
I20240502 07:59:42.297822 21413 consensus_queue.cc:260] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:42.297999 21405 raft_consensus.cc:348] T 4ce741ca5e264ccfbdde163add361886 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.298084 21413 ts_tablet_manager.cc:1398] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:42.298117 21405 raft_consensus.cc:374] T 4ce741ca5e264ccfbdde163add361886 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:42.298148 21405 raft_consensus.cc:729] T 4ce741ca5e264ccfbdde163add361886 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c15e23a34059456788f9775031e27ec0, State: Initialized, Role: FOLLOWER
I20240502 07:59:42.298245 21407 consensus_queue.cc:260] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:42.298262 21405 consensus_queue.cc:260] T 4ce741ca5e264ccfbdde163add361886 P c15e23a34059456788f9775031e27ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.298496 21407 ts_tablet_manager.cc:1398] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:42.298519 21405 ts_tablet_manager.cc:1398] T 4ce741ca5e264ccfbdde163add361886 P c15e23a34059456788f9775031e27ec0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:42.298676 21405 tablet_bootstrap.cc:492] T 1bc5937510214c3da7731d3ad2a67842 P c15e23a34059456788f9775031e27ec0: Bootstrap starting.
I20240502 07:59:42.299480 21405 tablet_bootstrap.cc:654] T 1bc5937510214c3da7731d3ad2a67842 P c15e23a34059456788f9775031e27ec0: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:42.300485 21405 tablet_bootstrap.cc:492] T 1bc5937510214c3da7731d3ad2a67842 P c15e23a34059456788f9775031e27ec0: No bootstrap required, opened a new log
I20240502 07:59:42.300567 21405 ts_tablet_manager.cc:1367] T 1bc5937510214c3da7731d3ad2a67842 P c15e23a34059456788f9775031e27ec0: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 07:59:42.300843 21545 raft_consensus.cc:482] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:42.300925 21545 raft_consensus.cc:504] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.301445 21408 tablet_bootstrap.cc:492] T 1453e833d0ad494583c44b5496088f6d P 29a7180fe4574c09a8dce7c27500aab2: No bootstrap required, opened a new log
I20240502 07:59:42.301535 21408 ts_tablet_manager.cc:1367] T 1453e833d0ad494583c44b5496088f6d P 29a7180fe4574c09a8dce7c27500aab2: Time spent bootstrapping tablet: real 0.039s user 0.001s sys 0.000s
I20240502 07:59:42.301527 21545 leader_election.cc:288] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703), b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331), 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587), b443e68b6212450286869124684bdfff (127.3.139.69:37923), c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085), b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577)
I20240502 07:59:42.301733 21405 raft_consensus.cc:348] T 1bc5937510214c3da7731d3ad2a67842 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:42.301831 21405 raft_consensus.cc:374] T 1bc5937510214c3da7731d3ad2a67842 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:42.301860 21405 raft_consensus.cc:729] T 1bc5937510214c3da7731d3ad2a67842 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c15e23a34059456788f9775031e27ec0, State: Initialized, Role: FOLLOWER
I20240502 07:59:42.301947 20802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ce741ca5e264ccfbdde163add361886" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15e23a34059456788f9775031e27ec0" is_pre_election: true
I20240502 07:59:42.301956 21408 raft_consensus.cc:348] T 1453e833d0ad494583c44b5496088f6d P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.301992 21405 consensus_queue.cc:260] T 1bc5937510214c3da7731d3ad2a67842 P c15e23a34059456788f9775031e27ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:42.302069 21408 raft_consensus.cc:374] T 1453e833d0ad494583c44b5496088f6d P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:42.302057 20802 raft_consensus.cc:2454] T 4ce741ca5e264ccfbdde163add361886 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e951f009258e473cb57cf8773c7975d4 in term 0.
I20240502 07:59:42.302099 21408 raft_consensus.cc:729] T 1453e833d0ad494583c44b5496088f6d P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29a7180fe4574c09a8dce7c27500aab2, State: Initialized, Role: FOLLOWER
I20240502 07:59:42.302251 21405 ts_tablet_manager.cc:1398] T 1bc5937510214c3da7731d3ad2a67842 P c15e23a34059456788f9775031e27ec0: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 07:59:42.302206 20274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ce741ca5e264ccfbdde163add361886" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" is_pre_election: true
I20240502 07:59:42.302218 21408 consensus_queue.cc:260] T 1453e833d0ad494583c44b5496088f6d P 29a7180fe4574c09a8dce7c27500aab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.302320 20274 raft_consensus.cc:2454] T 4ce741ca5e264ccfbdde163add361886 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e951f009258e473cb57cf8773c7975d4 in term 0.
I20240502 07:59:42.302451 21408 ts_tablet_manager.cc:1398] T 1453e833d0ad494583c44b5496088f6d P 29a7180fe4574c09a8dce7c27500aab2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:42.302594 21408 tablet_bootstrap.cc:492] T 1bc5937510214c3da7731d3ad2a67842 P 29a7180fe4574c09a8dce7c27500aab2: Bootstrap starting.
I20240502 07:59:42.302857 20407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ce741ca5e264ccfbdde163add361886" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35642aa2099c40e59bc2092604078e07" is_pre_election: true
I20240502 07:59:42.302975 20407 raft_consensus.cc:2454] T 4ce741ca5e264ccfbdde163add361886 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e951f009258e473cb57cf8773c7975d4 in term 0.
I20240502 07:59:42.303423 21408 tablet_bootstrap.cc:654] T 1bc5937510214c3da7731d3ad2a67842 P 29a7180fe4574c09a8dce7c27500aab2: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:42.303509 20474 leader_election.cc:302] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 35642aa2099c40e59bc2092604078e07, b7604c0eb3bf4a7cb260b564a94a1191, c15e23a34059456788f9775031e27ec0, e951f009258e473cb57cf8773c7975d4; no voters:
I20240502 07:59:42.303687 21545 raft_consensus.cc:2781] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:42.303736 21545 raft_consensus.cc:482] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:42.303769 21545 raft_consensus.cc:3037] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:42.304132 20669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ce741ca5e264ccfbdde163add361886" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b443e68b6212450286869124684bdfff" is_pre_election: true
I20240502 07:59:42.304244 20669 raft_consensus.cc:2454] T 4ce741ca5e264ccfbdde163add361886 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e951f009258e473cb57cf8773c7975d4 in term 0.
I20240502 07:59:42.304697 20141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ce741ca5e264ccfbdde163add361886" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7180fe4574c09a8dce7c27500aab2" is_pre_election: true
I20240502 07:59:42.304764 21545 raft_consensus.cc:504] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.305078 20935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ce741ca5e264ccfbdde163add361886" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f471f8b5944986a7d878cf6ac550b8" is_pre_election: true
I20240502 07:59:42.305217 20935 raft_consensus.cc:2454] T 4ce741ca5e264ccfbdde163add361886 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e951f009258e473cb57cf8773c7975d4 in term 0.
I20240502 07:59:42.305295 21545 leader_election.cc:288] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [CANDIDATE]: Term 1 election: Requested vote from peers 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703), b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331), 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587), b443e68b6212450286869124684bdfff (127.3.139.69:37923), c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085), b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577)
W20240502 07:59:42.305653 20473 leader_election.cc:341] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703): Illegal state: must be running to vote when last-logged opid is not known
I20240502 07:59:42.306092 20141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ce741ca5e264ccfbdde163add361886" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7180fe4574c09a8dce7c27500aab2"
I20240502 07:59:42.306160 21408 tablet_bootstrap.cc:492] T 1bc5937510214c3da7731d3ad2a67842 P 29a7180fe4574c09a8dce7c27500aab2: No bootstrap required, opened a new log
I20240502 07:59:42.306233 21408 ts_tablet_manager.cc:1367] T 1bc5937510214c3da7731d3ad2a67842 P 29a7180fe4574c09a8dce7c27500aab2: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 07:59:42.306457 20802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ce741ca5e264ccfbdde163add361886" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15e23a34059456788f9775031e27ec0"
I20240502 07:59:42.306567 20802 raft_consensus.cc:3037] T 4ce741ca5e264ccfbdde163add361886 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:42.306862 21408 raft_consensus.cc:348] T 1bc5937510214c3da7731d3ad2a67842 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:42.306960 21408 raft_consensus.cc:374] T 1bc5937510214c3da7731d3ad2a67842 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:42.306988 21408 raft_consensus.cc:729] T 1bc5937510214c3da7731d3ad2a67842 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29a7180fe4574c09a8dce7c27500aab2, State: Initialized, Role: FOLLOWER
I20240502 07:59:42.307116 21408 consensus_queue.cc:260] T 1bc5937510214c3da7731d3ad2a67842 P 29a7180fe4574c09a8dce7c27500aab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:42.307391 21408 ts_tablet_manager.cc:1398] T 1bc5937510214c3da7731d3ad2a67842 P 29a7180fe4574c09a8dce7c27500aab2: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:42.307489 21423 raft_consensus.cc:482] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:42.307534 20802 raft_consensus.cc:2454] T 4ce741ca5e264ccfbdde163add361886 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e951f009258e473cb57cf8773c7975d4 in term 1.
I20240502 07:59:42.307571 21408 tablet_bootstrap.cc:492] T 4ce741ca5e264ccfbdde163add361886 P 29a7180fe4574c09a8dce7c27500aab2: Bootstrap starting.
I20240502 07:59:42.307577 21423 raft_consensus.cc:504] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:42.308032 20669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ce741ca5e264ccfbdde163add361886" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b443e68b6212450286869124684bdfff"
I20240502 07:59:42.308120 20669 raft_consensus.cc:3037] T 4ce741ca5e264ccfbdde163add361886 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:42.308125 20274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ce741ca5e264ccfbdde163add361886" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7604c0eb3bf4a7cb260b564a94a1191"
I20240502 07:59:42.308230 20274 raft_consensus.cc:3037] T 4ce741ca5e264ccfbdde163add361886 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:42.308421 21408 tablet_bootstrap.cc:654] T 4ce741ca5e264ccfbdde163add361886 P 29a7180fe4574c09a8dce7c27500aab2: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:42.309051 21423 leader_election.cc:288] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315), c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085), b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577), b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331), 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703), 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587)
I20240502 07:59:42.309167 20274 raft_consensus.cc:2454] T 4ce741ca5e264ccfbdde163add361886 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e951f009258e473cb57cf8773c7975d4 in term 1.
I20240502 07:59:42.309842 20935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ce741ca5e264ccfbdde163add361886" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f471f8b5944986a7d878cf6ac550b8"
I20240502 07:59:42.309962 20935 raft_consensus.cc:3037] T 4ce741ca5e264ccfbdde163add361886 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:42.310901 20935 raft_consensus.cc:2454] T 4ce741ca5e264ccfbdde163add361886 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e951f009258e473cb57cf8773c7975d4 in term 1.
W20240502 07:59:42.311268 20473 leader_election.cc:341] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703): Illegal state: must be running to vote when last-logged opid is not known
I20240502 07:59:42.311353 20407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ce741ca5e264ccfbdde163add361886" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35642aa2099c40e59bc2092604078e07"
I20240502 07:59:42.311457 20407 raft_consensus.cc:3037] T 4ce741ca5e264ccfbdde163add361886 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:42.309056 20669 raft_consensus.cc:2454] T 4ce741ca5e264ccfbdde163add361886 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e951f009258e473cb57cf8773c7975d4 in term 1.
I20240502 07:59:42.312358 20407 raft_consensus.cc:2454] T 4ce741ca5e264ccfbdde163add361886 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e951f009258e473cb57cf8773c7975d4 in term 1.
I20240502 07:59:42.312816 20475 leader_election.cc:302] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 35642aa2099c40e59bc2092604078e07, b443e68b6212450286869124684bdfff, b7604c0eb3bf4a7cb260b564a94a1191, e951f009258e473cb57cf8773c7975d4; no voters: 29a7180fe4574c09a8dce7c27500aab2
I20240502 07:59:42.313194 21545 raft_consensus.cc:2781] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:42.313285 21545 raft_consensus.cc:686] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [term 1 LEADER]: Becoming Leader. State: Replica: e951f009258e473cb57cf8773c7975d4, State: Running, Role: LEADER
I20240502 07:59:42.313417 21545 consensus_queue.cc:237] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.314337 20802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bc5937510214c3da7731d3ad2a67842" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15e23a34059456788f9775031e27ec0" is_pre_election: true
I20240502 07:59:42.314458 20802 raft_consensus.cc:2454] T 1bc5937510214c3da7731d3ad2a67842 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b443e68b6212450286869124684bdfff in term 0.
I20240502 07:59:42.316213 19998 catalog_manager.cc:5484] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to e951f009258e473cb57cf8773c7975d4 (127.3.139.68). New cstate: current_term: 1 leader_uuid: "e951f009258e473cb57cf8773c7975d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:42.317706 20538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bc5937510214c3da7731d3ad2a67842" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e951f009258e473cb57cf8773c7975d4" is_pre_election: true
I20240502 07:59:42.317819 20538 raft_consensus.cc:2454] T 1bc5937510214c3da7731d3ad2a67842 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b443e68b6212450286869124684bdfff in term 0.
I20240502 07:59:42.319548 20274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bc5937510214c3da7731d3ad2a67842" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" is_pre_election: true
I20240502 07:59:42.319689 20274 raft_consensus.cc:2454] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b443e68b6212450286869124684bdfff in term 0.
I20240502 07:59:42.319787 20935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bc5937510214c3da7731d3ad2a67842" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f471f8b5944986a7d878cf6ac550b8" is_pre_election: true
I20240502 07:59:42.319895 20935 raft_consensus.cc:2454] T 1bc5937510214c3da7731d3ad2a67842 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b443e68b6212450286869124684bdfff in term 0.
I20240502 07:59:42.320037 20606 leader_election.cc:302] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: b443e68b6212450286869124684bdfff, b7604c0eb3bf4a7cb260b564a94a1191, c15e23a34059456788f9775031e27ec0, e951f009258e473cb57cf8773c7975d4; no voters:
I20240502 07:59:42.320173 20407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bc5937510214c3da7731d3ad2a67842" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35642aa2099c40e59bc2092604078e07" is_pre_election: true
I20240502 07:59:42.320286 21423 raft_consensus.cc:2781] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:42.320303 20407 raft_consensus.cc:2454] T 1bc5937510214c3da7731d3ad2a67842 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b443e68b6212450286869124684bdfff in term 0.
I20240502 07:59:42.320335 21423 raft_consensus.cc:482] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:42.320369 21423 raft_consensus.cc:3037] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:42.320845 20141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bc5937510214c3da7731d3ad2a67842" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7180fe4574c09a8dce7c27500aab2" is_pre_election: true
I20240502 07:59:42.320966 20141 raft_consensus.cc:2454] T 1bc5937510214c3da7731d3ad2a67842 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b443e68b6212450286869124684bdfff in term 0.
I20240502 07:59:42.321260 21408 tablet_bootstrap.cc:492] T 4ce741ca5e264ccfbdde163add361886 P 29a7180fe4574c09a8dce7c27500aab2: No bootstrap required, opened a new log
I20240502 07:59:42.321240 21423 raft_consensus.cc:504] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:42.321341 21408 ts_tablet_manager.cc:1367] T 4ce741ca5e264ccfbdde163add361886 P 29a7180fe4574c09a8dce7c27500aab2: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.001s
I20240502 07:59:42.321697 21423 leader_election.cc:288] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [CANDIDATE]: Term 1 election: Requested vote from peers e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315), c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085), b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577), b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331), 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703), 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587)
I20240502 07:59:42.321796 21408 raft_consensus.cc:348] T 4ce741ca5e264ccfbdde163add361886 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.321906 21408 raft_consensus.cc:374] T 4ce741ca5e264ccfbdde163add361886 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:42.321938 21408 raft_consensus.cc:729] T 4ce741ca5e264ccfbdde163add361886 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29a7180fe4574c09a8dce7c27500aab2, State: Initialized, Role: FOLLOWER
I20240502 07:59:42.322016 20274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bc5937510214c3da7731d3ad2a67842" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7604c0eb3bf4a7cb260b564a94a1191"
I20240502 07:59:42.322050 21408 consensus_queue.cc:260] T 4ce741ca5e264ccfbdde163add361886 P 29a7180fe4574c09a8dce7c27500aab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.322081 20802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bc5937510214c3da7731d3ad2a67842" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15e23a34059456788f9775031e27ec0"
I20240502 07:59:42.322108 20274 raft_consensus.cc:3037] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:42.322098 20538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bc5937510214c3da7731d3ad2a67842" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e951f009258e473cb57cf8773c7975d4"
I20240502 07:59:42.322168 20802 raft_consensus.cc:3037] T 1bc5937510214c3da7731d3ad2a67842 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:42.322185 20538 raft_consensus.cc:3037] T 1bc5937510214c3da7731d3ad2a67842 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:42.322319 21408 ts_tablet_manager.cc:1398] T 4ce741ca5e264ccfbdde163add361886 P 29a7180fe4574c09a8dce7c27500aab2: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:42.322930 20802 raft_consensus.cc:2454] T 1bc5937510214c3da7731d3ad2a67842 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b443e68b6212450286869124684bdfff in term 1.
I20240502 07:59:42.322957 20274 raft_consensus.cc:2454] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b443e68b6212450286869124684bdfff in term 1.
I20240502 07:59:42.322997 20538 raft_consensus.cc:2454] T 1bc5937510214c3da7731d3ad2a67842 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b443e68b6212450286869124684bdfff in term 1.
I20240502 07:59:42.323225 20935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bc5937510214c3da7731d3ad2a67842" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f471f8b5944986a7d878cf6ac550b8"
I20240502 07:59:42.323334 20935 raft_consensus.cc:3037] T 1bc5937510214c3da7731d3ad2a67842 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:42.323782 20605 leader_election.cc:302] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: b443e68b6212450286869124684bdfff, b7604c0eb3bf4a7cb260b564a94a1191, c15e23a34059456788f9775031e27ec0, e951f009258e473cb57cf8773c7975d4; no voters:
I20240502 07:59:42.323953 21423 raft_consensus.cc:2781] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:42.324128 20935 raft_consensus.cc:2454] T 1bc5937510214c3da7731d3ad2a67842 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b443e68b6212450286869124684bdfff in term 1.
I20240502 07:59:42.324144 21423 raft_consensus.cc:686] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [term 1 LEADER]: Becoming Leader. State: Replica: b443e68b6212450286869124684bdfff, State: Running, Role: LEADER
I20240502 07:59:42.324364 20407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bc5937510214c3da7731d3ad2a67842" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35642aa2099c40e59bc2092604078e07"
I20240502 07:59:42.324473 20407 raft_consensus.cc:3037] T 1bc5937510214c3da7731d3ad2a67842 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:42.324451 21423 consensus_queue.cc:237] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:42.325331 20407 raft_consensus.cc:2454] T 1bc5937510214c3da7731d3ad2a67842 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b443e68b6212450286869124684bdfff in term 1.
I20240502 07:59:42.325655 20141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bc5937510214c3da7731d3ad2a67842" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7180fe4574c09a8dce7c27500aab2"
I20240502 07:59:42.325775 20141 raft_consensus.cc:3037] T 1bc5937510214c3da7731d3ad2a67842 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:42.326663 20141 raft_consensus.cc:2454] T 1bc5937510214c3da7731d3ad2a67842 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b443e68b6212450286869124684bdfff in term 1.
I20240502 07:59:42.327286 19998 catalog_manager.cc:5484] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff reported cstate change: term changed from 0 to 1, leader changed from <none> to b443e68b6212450286869124684bdfff (127.3.139.69). New cstate: current_term: 1 leader_uuid: "b443e68b6212450286869124684bdfff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:42.534888 21456 raft_consensus.cc:482] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:42.535015 21456 raft_consensus.cc:504] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.535642 21456 leader_election.cc:288] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703), b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331), 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587), c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085), e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315), b443e68b6212450286869124684bdfff (127.3.139.69:37923)
I20240502 07:59:42.535883 20274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1453e833d0ad494583c44b5496088f6d" candidate_uuid: "b5f471f8b5944986a7d878cf6ac550b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" is_pre_election: true
I20240502 07:59:42.535933 20407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1453e833d0ad494583c44b5496088f6d" candidate_uuid: "b5f471f8b5944986a7d878cf6ac550b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35642aa2099c40e59bc2092604078e07" is_pre_election: true
I20240502 07:59:42.536026 20274 raft_consensus.cc:2454] T 1453e833d0ad494583c44b5496088f6d P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5f471f8b5944986a7d878cf6ac550b8 in term 0.
I20240502 07:59:42.536020 20538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1453e833d0ad494583c44b5496088f6d" candidate_uuid: "b5f471f8b5944986a7d878cf6ac550b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e951f009258e473cb57cf8773c7975d4" is_pre_election: true
I20240502 07:59:42.536087 20407 raft_consensus.cc:2454] T 1453e833d0ad494583c44b5496088f6d P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5f471f8b5944986a7d878cf6ac550b8 in term 0.
I20240502 07:59:42.536144 20538 raft_consensus.cc:2454] T 1453e833d0ad494583c44b5496088f6d P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5f471f8b5944986a7d878cf6ac550b8 in term 0.
I20240502 07:59:42.536180 20802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1453e833d0ad494583c44b5496088f6d" candidate_uuid: "b5f471f8b5944986a7d878cf6ac550b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15e23a34059456788f9775031e27ec0" is_pre_election: true
I20240502 07:59:42.536319 20802 raft_consensus.cc:2454] T 1453e833d0ad494583c44b5496088f6d P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5f471f8b5944986a7d878cf6ac550b8 in term 0.
I20240502 07:59:42.536361 20141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1453e833d0ad494583c44b5496088f6d" candidate_uuid: "b5f471f8b5944986a7d878cf6ac550b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7180fe4574c09a8dce7c27500aab2" is_pre_election: true
I20240502 07:59:42.536494 20141 raft_consensus.cc:2454] T 1453e833d0ad494583c44b5496088f6d P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5f471f8b5944986a7d878cf6ac550b8 in term 0.
I20240502 07:59:42.536631 20869 leader_election.cc:302] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 35642aa2099c40e59bc2092604078e07, b5f471f8b5944986a7d878cf6ac550b8, b7604c0eb3bf4a7cb260b564a94a1191, c15e23a34059456788f9775031e27ec0; no voters:
I20240502 07:59:42.536650 20669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1453e833d0ad494583c44b5496088f6d" candidate_uuid: "b5f471f8b5944986a7d878cf6ac550b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b443e68b6212450286869124684bdfff" is_pre_election: true
I20240502 07:59:42.536792 20669 raft_consensus.cc:2454] T 1453e833d0ad494583c44b5496088f6d P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5f471f8b5944986a7d878cf6ac550b8 in term 0.
I20240502 07:59:42.536821 21456 raft_consensus.cc:2781] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:42.536866 21456 raft_consensus.cc:482] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:42.536897 21456 raft_consensus.cc:3037] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:42.537715 21456 raft_consensus.cc:504] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.538215 21456 leader_election.cc:288] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [CANDIDATE]: Term 1 election: Requested vote from peers 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703), b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331), 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587), c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085), e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315), b443e68b6212450286869124684bdfff (127.3.139.69:37923)
I20240502 07:59:42.538491 20275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1453e833d0ad494583c44b5496088f6d" candidate_uuid: "b5f471f8b5944986a7d878cf6ac550b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7604c0eb3bf4a7cb260b564a94a1191"
I20240502 07:59:42.538502 20538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1453e833d0ad494583c44b5496088f6d" candidate_uuid: "b5f471f8b5944986a7d878cf6ac550b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e951f009258e473cb57cf8773c7975d4"
I20240502 07:59:42.538586 20275 raft_consensus.cc:3037] T 1453e833d0ad494583c44b5496088f6d P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:42.538599 20538 raft_consensus.cc:3037] T 1453e833d0ad494583c44b5496088f6d P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:42.538625 20802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1453e833d0ad494583c44b5496088f6d" candidate_uuid: "b5f471f8b5944986a7d878cf6ac550b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15e23a34059456788f9775031e27ec0"
I20240502 07:59:42.538707 20802 raft_consensus.cc:3037] T 1453e833d0ad494583c44b5496088f6d P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:42.539101 20669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1453e833d0ad494583c44b5496088f6d" candidate_uuid: "b5f471f8b5944986a7d878cf6ac550b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b443e68b6212450286869124684bdfff"
I20240502 07:59:42.539206 20669 raft_consensus.cc:3037] T 1453e833d0ad494583c44b5496088f6d P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:42.539517 20275 raft_consensus.cc:2454] T 1453e833d0ad494583c44b5496088f6d P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5f471f8b5944986a7d878cf6ac550b8 in term 1.
I20240502 07:59:42.539572 20538 raft_consensus.cc:2454] T 1453e833d0ad494583c44b5496088f6d P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5f471f8b5944986a7d878cf6ac550b8 in term 1.
I20240502 07:59:42.539572 20802 raft_consensus.cc:2454] T 1453e833d0ad494583c44b5496088f6d P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5f471f8b5944986a7d878cf6ac550b8 in term 1.
I20240502 07:59:42.539832 20141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1453e833d0ad494583c44b5496088f6d" candidate_uuid: "b5f471f8b5944986a7d878cf6ac550b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7180fe4574c09a8dce7c27500aab2"
I20240502 07:59:42.539943 20141 raft_consensus.cc:3037] T 1453e833d0ad494583c44b5496088f6d P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:42.539963 20407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1453e833d0ad494583c44b5496088f6d" candidate_uuid: "b5f471f8b5944986a7d878cf6ac550b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35642aa2099c40e59bc2092604078e07"
I20240502 07:59:42.540083 20407 raft_consensus.cc:3037] T 1453e833d0ad494583c44b5496088f6d P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:42.540132 20669 raft_consensus.cc:2454] T 1453e833d0ad494583c44b5496088f6d P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5f471f8b5944986a7d878cf6ac550b8 in term 1.
I20240502 07:59:42.540726 20141 raft_consensus.cc:2454] T 1453e833d0ad494583c44b5496088f6d P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5f471f8b5944986a7d878cf6ac550b8 in term 1.
I20240502 07:59:42.540853 20407 raft_consensus.cc:2454] T 1453e833d0ad494583c44b5496088f6d P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5f471f8b5944986a7d878cf6ac550b8 in term 1.
I20240502 07:59:42.541112 20871 leader_election.cc:302] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 35642aa2099c40e59bc2092604078e07, b443e68b6212450286869124684bdfff, b5f471f8b5944986a7d878cf6ac550b8, b7604c0eb3bf4a7cb260b564a94a1191; no voters:
I20240502 07:59:42.541265 21456 raft_consensus.cc:2781] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:42.541337 21456 raft_consensus.cc:686] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [term 1 LEADER]: Becoming Leader. State: Replica: b5f471f8b5944986a7d878cf6ac550b8, State: Running, Role: LEADER
I20240502 07:59:42.541478 21456 consensus_queue.cc:237] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.543512 19998 catalog_manager.cc:5484] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 reported cstate change: term changed from 0 to 1, leader changed from <none> to b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71). New cstate: current_term: 1 leader_uuid: "b5f471f8b5944986a7d878cf6ac550b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: HEALTHY } } }
W20240502 07:59:42.678592 21603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:42.678802 21603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:42.681828 21603 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_1'
I20240502 07:59:42.716892 21545 consensus_queue.cc:1035] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:42.738507 21423 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:42.766548 20935 raft_consensus.cc:1261] T 4ce741ca5e264ccfbdde163add361886 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:42.766996 20802 raft_consensus.cc:1261] T 4ce741ca5e264ccfbdde163add361886 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:42.768113 21545 consensus_queue.cc:1035] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:42.768743 21545 consensus_queue.cc:1035] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:42.770506 20407 raft_consensus.cc:1261] T 4ce741ca5e264ccfbdde163add361886 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:42.770928 21428 raft_consensus.cc:482] T 4ce741ca5e264ccfbdde163add361886 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:42.771071 21428 raft_consensus.cc:504] T 4ce741ca5e264ccfbdde163add361886 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.771659 20407 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:42.772053 20275 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:42.772042 21423 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:42.772429 21423 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:42.772656 21428 leader_election.cc:288] T 4ce741ca5e264ccfbdde163add361886 P 29a7180fe4574c09a8dce7c27500aab2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331), 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587), b443e68b6212450286869124684bdfff (127.3.139.69:37923), e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315), c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085), b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577)
I20240502 07:59:42.772842 21545 consensus_queue.cc:1035] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:42.775085 20274 raft_consensus.cc:1261] T 4ce741ca5e264ccfbdde163add361886 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:42.775036 20141 raft_consensus.cc:3037] T 4ce741ca5e264ccfbdde163add361886 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:42.775451 21545 consensus_queue.cc:1035] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:42.776255 20141 raft_consensus.cc:1261] T 4ce741ca5e264ccfbdde163add361886 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:42.776607 21545 consensus_queue.cc:1035] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:42.777617 20143 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:42.778306 21600 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:42.781731 20538 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:42.781736 20803 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:42.782352 21600 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:42.782617 21600 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:42.791611 20274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ce741ca5e264ccfbdde163add361886" candidate_uuid: "29a7180fe4574c09a8dce7c27500aab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" is_pre_election: true
I20240502 07:59:42.794312 20407 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:42.794823 20142 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:42.795172 20539 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:42.795290 20802 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:42.795409 20273 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:42.796015 21503 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:42.796314 21503 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:42.796572 21503 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:42.796777 21503 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:42.796809 20669 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:42.799839 21456 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:42.800081 21456 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:42.809563 20537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ce741ca5e264ccfbdde163add361886" candidate_uuid: "29a7180fe4574c09a8dce7c27500aab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e951f009258e473cb57cf8773c7975d4" is_pre_election: true
I20240502 07:59:42.825459 20800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ce741ca5e264ccfbdde163add361886" candidate_uuid: "29a7180fe4574c09a8dce7c27500aab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15e23a34059456788f9775031e27ec0" is_pre_election: true
I20240502 07:59:42.851575 20935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ce741ca5e264ccfbdde163add361886" candidate_uuid: "29a7180fe4574c09a8dce7c27500aab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f471f8b5944986a7d878cf6ac550b8" is_pre_election: true
I20240502 07:59:42.852001 20079 leader_election.cc:302] T 4ce741ca5e264ccfbdde163add361886 P 29a7180fe4574c09a8dce7c27500aab2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 29a7180fe4574c09a8dce7c27500aab2; no voters: b5f471f8b5944986a7d878cf6ac550b8, b7604c0eb3bf4a7cb260b564a94a1191, c15e23a34059456788f9775031e27ec0, e951f009258e473cb57cf8773c7975d4
I20240502 07:59:42.852279 21428 raft_consensus.cc:2726] T 4ce741ca5e264ccfbdde163add361886 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 07:59:42.856290 20407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ce741ca5e264ccfbdde163add361886" candidate_uuid: "29a7180fe4574c09a8dce7c27500aab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35642aa2099c40e59bc2092604078e07" is_pre_election: true
I20240502 07:59:42.886765 20669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ce741ca5e264ccfbdde163add361886" candidate_uuid: "29a7180fe4574c09a8dce7c27500aab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b443e68b6212450286869124684bdfff" is_pre_election: true
INSERT report
rows total: 2000
time total: 215.034 ms
time per row: 0.107517 ms
I20240502 07:59:42.934270 20001 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33518:
name: "rebalance_test_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 7
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240502 07:59:42.934705 20001 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_2 in case of a server failure: 8 tablet servers would be needed, 7 are available. Consider bringing up more tablet servers.
I20240502 07:59:42.939594 20518 tablet_service.cc:1467] Processing CreateTablet for tablet dbf4b90cac9e4ff3b2936a1fa2ca8561 (DEFAULT_TABLE table=rebalance_test_table_2 [id=30a30b02a2404507a95fc3b5bb77bdd1]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:42.939736 20783 tablet_service.cc:1467] Processing CreateTablet for tablet dbf4b90cac9e4ff3b2936a1fa2ca8561 (DEFAULT_TABLE table=rebalance_test_table_2 [id=30a30b02a2404507a95fc3b5bb77bdd1]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:42.939930 20518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbf4b90cac9e4ff3b2936a1fa2ca8561. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:42.940042 20783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbf4b90cac9e4ff3b2936a1fa2ca8561. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:42.940057 20122 tablet_service.cc:1467] Processing CreateTablet for tablet dbf4b90cac9e4ff3b2936a1fa2ca8561 (DEFAULT_TABLE table=rebalance_test_table_2 [id=30a30b02a2404507a95fc3b5bb77bdd1]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:42.940308 20122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbf4b90cac9e4ff3b2936a1fa2ca8561. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:42.942117 21708 tablet_bootstrap.cc:492] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P c15e23a34059456788f9775031e27ec0: Bootstrap starting.
I20240502 07:59:42.942323 20649 tablet_service.cc:1467] Processing CreateTablet for tablet dbf4b90cac9e4ff3b2936a1fa2ca8561 (DEFAULT_TABLE table=rebalance_test_table_2 [id=30a30b02a2404507a95fc3b5bb77bdd1]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:42.942600 20649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbf4b90cac9e4ff3b2936a1fa2ca8561. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:42.942581 20386 tablet_service.cc:1467] Processing CreateTablet for tablet dbf4b90cac9e4ff3b2936a1fa2ca8561 (DEFAULT_TABLE table=rebalance_test_table_2 [id=30a30b02a2404507a95fc3b5bb77bdd1]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:42.942842 20386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbf4b90cac9e4ff3b2936a1fa2ca8561. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:42.943189 20123 tablet_service.cc:1467] Processing CreateTablet for tablet 880876d792554606a7ccf21f9ec08e0f (DEFAULT_TABLE table=rebalance_test_table_2 [id=30a30b02a2404507a95fc3b5bb77bdd1]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:42.943361 21708 tablet_bootstrap.cc:654] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P c15e23a34059456788f9775031e27ec0: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:42.943429 20123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 880876d792554606a7ccf21f9ec08e0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:42.943830 20254 tablet_service.cc:1467] Processing CreateTablet for tablet dbf4b90cac9e4ff3b2936a1fa2ca8561 (DEFAULT_TABLE table=rebalance_test_table_2 [id=30a30b02a2404507a95fc3b5bb77bdd1]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:42.944082 20254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbf4b90cac9e4ff3b2936a1fa2ca8561. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:42.945048 20387 tablet_service.cc:1467] Processing CreateTablet for tablet f588a3ec49a24b1a9a9e19cdf8c30ca5 (DEFAULT_TABLE table=rebalance_test_table_2 [id=30a30b02a2404507a95fc3b5bb77bdd1]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:42.945258 20387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f588a3ec49a24b1a9a9e19cdf8c30ca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:42.945397 20255 tablet_service.cc:1467] Processing CreateTablet for tablet 880876d792554606a7ccf21f9ec08e0f (DEFAULT_TABLE table=rebalance_test_table_2 [id=30a30b02a2404507a95fc3b5bb77bdd1]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:42.945616 20255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 880876d792554606a7ccf21f9ec08e0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:42.946154 20518 tablet_service.cc:1467] Processing CreateTablet for tablet f588a3ec49a24b1a9a9e19cdf8c30ca5 (DEFAULT_TABLE table=rebalance_test_table_2 [id=30a30b02a2404507a95fc3b5bb77bdd1]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:42.946309 20253 tablet_service.cc:1467] Processing CreateTablet for tablet f588a3ec49a24b1a9a9e19cdf8c30ca5 (DEFAULT_TABLE table=rebalance_test_table_2 [id=30a30b02a2404507a95fc3b5bb77bdd1]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:42.946372 20518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f588a3ec49a24b1a9a9e19cdf8c30ca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:42.946517 20253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f588a3ec49a24b1a9a9e19cdf8c30ca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:42.947100 20385 tablet_service.cc:1467] Processing CreateTablet for tablet 880876d792554606a7ccf21f9ec08e0f (DEFAULT_TABLE table=rebalance_test_table_2 [id=30a30b02a2404507a95fc3b5bb77bdd1]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:42.947309 20385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 880876d792554606a7ccf21f9ec08e0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:42.947598 20650 tablet_service.cc:1467] Processing CreateTablet for tablet 880876d792554606a7ccf21f9ec08e0f (DEFAULT_TABLE table=rebalance_test_table_2 [id=30a30b02a2404507a95fc3b5bb77bdd1]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:42.947809 20650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 880876d792554606a7ccf21f9ec08e0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:42.948537 21709 tablet_bootstrap.cc:492] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 29a7180fe4574c09a8dce7c27500aab2: Bootstrap starting.
I20240502 07:59:42.949167 20651 tablet_service.cc:1467] Processing CreateTablet for tablet f588a3ec49a24b1a9a9e19cdf8c30ca5 (DEFAULT_TABLE table=rebalance_test_table_2 [id=30a30b02a2404507a95fc3b5bb77bdd1]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:42.949338 21709 tablet_bootstrap.cc:654] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 29a7180fe4574c09a8dce7c27500aab2: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:42.949402 20651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f588a3ec49a24b1a9a9e19cdf8c30ca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:42.949786 21707 tablet_bootstrap.cc:492] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4: Bootstrap starting.
I20240502 07:59:42.949820 20519 tablet_service.cc:1467] Processing CreateTablet for tablet 880876d792554606a7ccf21f9ec08e0f (DEFAULT_TABLE table=rebalance_test_table_2 [id=30a30b02a2404507a95fc3b5bb77bdd1]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:42.950011 21711 tablet_bootstrap.cc:492] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b443e68b6212450286869124684bdfff: Bootstrap starting.
I20240502 07:59:42.950055 20519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 880876d792554606a7ccf21f9ec08e0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:42.950701 21707 tablet_bootstrap.cc:654] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:42.950853 21711 tablet_bootstrap.cc:654] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b443e68b6212450286869124684bdfff: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:42.951640 20913 tablet_service.cc:1467] Processing CreateTablet for tablet dbf4b90cac9e4ff3b2936a1fa2ca8561 (DEFAULT_TABLE table=rebalance_test_table_2 [id=30a30b02a2404507a95fc3b5bb77bdd1]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:42.951897 20913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbf4b90cac9e4ff3b2936a1fa2ca8561. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:42.952070 21709 tablet_bootstrap.cc:492] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 29a7180fe4574c09a8dce7c27500aab2: No bootstrap required, opened a new log
I20240502 07:59:42.952103 21711 tablet_bootstrap.cc:492] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b443e68b6212450286869124684bdfff: No bootstrap required, opened a new log
I20240502 07:59:42.952190 21711 ts_tablet_manager.cc:1367] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b443e68b6212450286869124684bdfff: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 07:59:42.952220 21709 ts_tablet_manager.cc:1367] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 29a7180fe4574c09a8dce7c27500aab2: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 07:59:42.952628 21711 raft_consensus.cc:348] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:42.952766 21711 raft_consensus.cc:374] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:42.952809 21711 raft_consensus.cc:729] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b443e68b6212450286869124684bdfff, State: Initialized, Role: FOLLOWER
I20240502 07:59:42.952724 21709 raft_consensus.cc:348] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:42.952950 21711 consensus_queue.cc:260] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b443e68b6212450286869124684bdfff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:42.953037 21709 raft_consensus.cc:374] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:42.953186 21709 raft_consensus.cc:729] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29a7180fe4574c09a8dce7c27500aab2, State: Initialized, Role: FOLLOWER
I20240502 07:59:42.953326 20122 tablet_service.cc:1467] Processing CreateTablet for tablet f588a3ec49a24b1a9a9e19cdf8c30ca5 (DEFAULT_TABLE table=rebalance_test_table_2 [id=30a30b02a2404507a95fc3b5bb77bdd1]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:42.953349 21709 consensus_queue.cc:260] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 29a7180fe4574c09a8dce7c27500aab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:42.953552 20122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f588a3ec49a24b1a9a9e19cdf8c30ca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:42.953773 21709 ts_tablet_manager.cc:1398] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 29a7180fe4574c09a8dce7c27500aab2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:42.953791 21711 ts_tablet_manager.cc:1398] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b443e68b6212450286869124684bdfff: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 07:59:42.953938 21711 tablet_bootstrap.cc:492] T 880876d792554606a7ccf21f9ec08e0f P b443e68b6212450286869124684bdfff: Bootstrap starting.
I20240502 07:59:42.954046 21709 tablet_bootstrap.cc:492] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2: Bootstrap starting.
I20240502 07:59:42.954115 20915 tablet_service.cc:1467] Processing CreateTablet for tablet f588a3ec49a24b1a9a9e19cdf8c30ca5 (DEFAULT_TABLE table=rebalance_test_table_2 [id=30a30b02a2404507a95fc3b5bb77bdd1]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:42.954335 20915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f588a3ec49a24b1a9a9e19cdf8c30ca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:42.954718 21711 tablet_bootstrap.cc:654] T 880876d792554606a7ccf21f9ec08e0f P b443e68b6212450286869124684bdfff: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:42.955323 21709 tablet_bootstrap.cc:654] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:42.955637 21711 tablet_bootstrap.cc:492] T 880876d792554606a7ccf21f9ec08e0f P b443e68b6212450286869124684bdfff: No bootstrap required, opened a new log
I20240502 07:59:42.955729 21711 ts_tablet_manager.cc:1367] T 880876d792554606a7ccf21f9ec08e0f P b443e68b6212450286869124684bdfff: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:42.956125 21711 raft_consensus.cc:348] T 880876d792554606a7ccf21f9ec08e0f P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.956245 21711 raft_consensus.cc:374] T 880876d792554606a7ccf21f9ec08e0f P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:42.956284 21711 raft_consensus.cc:729] T 880876d792554606a7ccf21f9ec08e0f P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b443e68b6212450286869124684bdfff, State: Initialized, Role: FOLLOWER
I20240502 07:59:42.956249 20914 tablet_service.cc:1467] Processing CreateTablet for tablet 880876d792554606a7ccf21f9ec08e0f (DEFAULT_TABLE table=rebalance_test_table_2 [id=30a30b02a2404507a95fc3b5bb77bdd1]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:42.956439 20914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 880876d792554606a7ccf21f9ec08e0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:42.956387 21711 consensus_queue.cc:260] T 880876d792554606a7ccf21f9ec08e0f P b443e68b6212450286869124684bdfff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.956624 21711 ts_tablet_manager.cc:1398] T 880876d792554606a7ccf21f9ec08e0f P b443e68b6212450286869124684bdfff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:42.956782 21711 tablet_bootstrap.cc:492] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b443e68b6212450286869124684bdfff: Bootstrap starting.
I20240502 07:59:42.957559 21711 tablet_bootstrap.cc:654] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b443e68b6212450286869124684bdfff: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:42.958309 20781 tablet_service.cc:1467] Processing CreateTablet for tablet 880876d792554606a7ccf21f9ec08e0f (DEFAULT_TABLE table=rebalance_test_table_2 [id=30a30b02a2404507a95fc3b5bb77bdd1]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:42.958544 20781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 880876d792554606a7ccf21f9ec08e0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:42.958557 21711 tablet_bootstrap.cc:492] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b443e68b6212450286869124684bdfff: No bootstrap required, opened a new log
I20240502 07:59:42.958629 21711 ts_tablet_manager.cc:1367] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b443e68b6212450286869124684bdfff: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240502 07:59:42.959017 20783 tablet_service.cc:1467] Processing CreateTablet for tablet f588a3ec49a24b1a9a9e19cdf8c30ca5 (DEFAULT_TABLE table=rebalance_test_table_2 [id=30a30b02a2404507a95fc3b5bb77bdd1]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:42.959148 21711 raft_consensus.cc:348] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.959241 20783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f588a3ec49a24b1a9a9e19cdf8c30ca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:42.959245 21711 raft_consensus.cc:374] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:42.959277 21711 raft_consensus.cc:729] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b443e68b6212450286869124684bdfff, State: Initialized, Role: FOLLOWER
I20240502 07:59:42.959379 21711 consensus_queue.cc:260] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b443e68b6212450286869124684bdfff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.959656 21711 ts_tablet_manager.cc:1398] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b443e68b6212450286869124684bdfff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:42.960976 21712 tablet_bootstrap.cc:492] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b7604c0eb3bf4a7cb260b564a94a1191: Bootstrap starting.
I20240502 07:59:42.961673 21712 tablet_bootstrap.cc:654] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b7604c0eb3bf4a7cb260b564a94a1191: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:42.962625 21708 tablet_bootstrap.cc:492] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P c15e23a34059456788f9775031e27ec0: No bootstrap required, opened a new log
I20240502 07:59:42.962625 21712 tablet_bootstrap.cc:492] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b7604c0eb3bf4a7cb260b564a94a1191: No bootstrap required, opened a new log
I20240502 07:59:42.962715 21708 ts_tablet_manager.cc:1367] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P c15e23a34059456788f9775031e27ec0: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.000s
I20240502 07:59:42.962805 21712 ts_tablet_manager.cc:1367] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b7604c0eb3bf4a7cb260b564a94a1191: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:42.963208 21717 tablet_bootstrap.cc:492] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b5f471f8b5944986a7d878cf6ac550b8: Bootstrap starting.
I20240502 07:59:42.963145 21708 raft_consensus.cc:348] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:42.963259 21708 raft_consensus.cc:374] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:42.963289 21708 raft_consensus.cc:729] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c15e23a34059456788f9775031e27ec0, State: Initialized, Role: FOLLOWER
I20240502 07:59:42.963251 21712 raft_consensus.cc:348] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:42.963377 21712 raft_consensus.cc:374] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:42.963407 21712 raft_consensus.cc:729] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7604c0eb3bf4a7cb260b564a94a1191, State: Initialized, Role: FOLLOWER
I20240502 07:59:42.963413 21708 consensus_queue.cc:260] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P c15e23a34059456788f9775031e27ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:42.963523 21712 consensus_queue.cc:260] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b7604c0eb3bf4a7cb260b564a94a1191 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:42.963765 21708 ts_tablet_manager.cc:1398] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P c15e23a34059456788f9775031e27ec0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:42.963783 21712 ts_tablet_manager.cc:1398] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b7604c0eb3bf4a7cb260b564a94a1191: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:42.964383 21712 tablet_bootstrap.cc:492] T 880876d792554606a7ccf21f9ec08e0f P b7604c0eb3bf4a7cb260b564a94a1191: Bootstrap starting.
I20240502 07:59:42.964653 21710 tablet_bootstrap.cc:492] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 35642aa2099c40e59bc2092604078e07: Bootstrap starting.
I20240502 07:59:42.965204 21712 tablet_bootstrap.cc:654] T 880876d792554606a7ccf21f9ec08e0f P b7604c0eb3bf4a7cb260b564a94a1191: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:42.965494 21710 tablet_bootstrap.cc:654] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 35642aa2099c40e59bc2092604078e07: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:42.966363 21708 tablet_bootstrap.cc:492] T 880876d792554606a7ccf21f9ec08e0f P c15e23a34059456788f9775031e27ec0: Bootstrap starting.
I20240502 07:59:42.966375 21717 tablet_bootstrap.cc:654] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b5f471f8b5944986a7d878cf6ac550b8: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:42.967010 21708 tablet_bootstrap.cc:654] T 880876d792554606a7ccf21f9ec08e0f P c15e23a34059456788f9775031e27ec0: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:42.967481 21717 tablet_bootstrap.cc:492] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b5f471f8b5944986a7d878cf6ac550b8: No bootstrap required, opened a new log
I20240502 07:59:42.967773 21710 tablet_bootstrap.cc:492] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 35642aa2099c40e59bc2092604078e07: No bootstrap required, opened a new log
I20240502 07:59:42.967855 21710 ts_tablet_manager.cc:1367] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 35642aa2099c40e59bc2092604078e07: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:42.968242 21710 raft_consensus.cc:348] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:42.968376 21710 raft_consensus.cc:374] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:42.968415 21710 raft_consensus.cc:729] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35642aa2099c40e59bc2092604078e07, State: Initialized, Role: FOLLOWER
I20240502 07:59:42.968560 21709 tablet_bootstrap.cc:492] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2: No bootstrap required, opened a new log
I20240502 07:59:42.968577 21717 ts_tablet_manager.cc:1367] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b5f471f8b5944986a7d878cf6ac550b8: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:59:42.968653 21709 ts_tablet_manager.cc:1367] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20240502 07:59:42.968773 21708 tablet_bootstrap.cc:492] T 880876d792554606a7ccf21f9ec08e0f P c15e23a34059456788f9775031e27ec0: No bootstrap required, opened a new log
I20240502 07:59:42.968852 21708 ts_tablet_manager.cc:1367] T 880876d792554606a7ccf21f9ec08e0f P c15e23a34059456788f9775031e27ec0: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 07:59:42.968986 21707 tablet_bootstrap.cc:492] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4: No bootstrap required, opened a new log
I20240502 07:59:42.968988 21717 raft_consensus.cc:348] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:42.969110 21717 raft_consensus.cc:374] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:42.969080 21709 raft_consensus.cc:348] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.969152 21717 raft_consensus.cc:729] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5f471f8b5944986a7d878cf6ac550b8, State: Initialized, Role: FOLLOWER
I20240502 07:59:42.969200 21709 raft_consensus.cc:374] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:42.969240 21709 raft_consensus.cc:729] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29a7180fe4574c09a8dce7c27500aab2, State: Initialized, Role: FOLLOWER
I20240502 07:59:42.969264 21708 raft_consensus.cc:348] T 880876d792554606a7ccf21f9ec08e0f P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.969357 21708 raft_consensus.cc:374] T 880876d792554606a7ccf21f9ec08e0f P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:42.969301 21717 consensus_queue.cc:260] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b5f471f8b5944986a7d878cf6ac550b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:42.969391 21708 raft_consensus.cc:729] T 880876d792554606a7ccf21f9ec08e0f P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c15e23a34059456788f9775031e27ec0, State: Initialized, Role: FOLLOWER
I20240502 07:59:42.969349 21709 consensus_queue.cc:260] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.969470 21707 ts_tablet_manager.cc:1367] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4: Time spent bootstrapping tablet: real 0.020s user 0.000s sys 0.002s
I20240502 07:59:42.969518 21708 consensus_queue.cc:260] T 880876d792554606a7ccf21f9ec08e0f P c15e23a34059456788f9775031e27ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.969628 21709 ts_tablet_manager.cc:1398] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:42.969827 21709 tablet_bootstrap.cc:492] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 29a7180fe4574c09a8dce7c27500aab2: Bootstrap starting.
I20240502 07:59:42.970122 21707 raft_consensus.cc:348] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:42.970302 21707 raft_consensus.cc:374] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:42.970384 21707 raft_consensus.cc:729] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e951f009258e473cb57cf8773c7975d4, State: Initialized, Role: FOLLOWER
I20240502 07:59:42.970577 21707 consensus_queue.cc:260] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:42.970655 21709 tablet_bootstrap.cc:654] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 29a7180fe4574c09a8dce7c27500aab2: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:42.971268 21708 ts_tablet_manager.cc:1398] T 880876d792554606a7ccf21f9ec08e0f P c15e23a34059456788f9775031e27ec0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:42.971400 21707 ts_tablet_manager.cc:1398] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 07:59:42.971578 21707 tablet_bootstrap.cc:492] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P e951f009258e473cb57cf8773c7975d4: Bootstrap starting.
I20240502 07:59:42.972007 21710 consensus_queue.cc:260] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 35642aa2099c40e59bc2092604078e07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:42.972638 21717 ts_tablet_manager.cc:1398] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b5f471f8b5944986a7d878cf6ac550b8: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:59:42.973074 21708 tablet_bootstrap.cc:492] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P c15e23a34059456788f9775031e27ec0: Bootstrap starting.
I20240502 07:59:42.973393 21707 tablet_bootstrap.cc:654] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P e951f009258e473cb57cf8773c7975d4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:42.973872 21708 tablet_bootstrap.cc:654] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P c15e23a34059456788f9775031e27ec0: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:42.974565 21717 tablet_bootstrap.cc:492] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8: Bootstrap starting.
I20240502 07:59:42.974615 21707 tablet_bootstrap.cc:492] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P e951f009258e473cb57cf8773c7975d4: No bootstrap required, opened a new log
I20240502 07:59:42.974697 21707 ts_tablet_manager.cc:1367] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P e951f009258e473cb57cf8773c7975d4: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 07:59:42.972860 21710 ts_tablet_manager.cc:1398] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 35642aa2099c40e59bc2092604078e07: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20240502 07:59:42.975122 21707 raft_consensus.cc:348] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.975232 21707 raft_consensus.cc:374] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:42.975270 21707 raft_consensus.cc:729] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e951f009258e473cb57cf8773c7975d4, State: Initialized, Role: FOLLOWER
I20240502 07:59:42.975297 21710 tablet_bootstrap.cc:492] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07: Bootstrap starting.
I20240502 07:59:42.975539 21707 consensus_queue.cc:260] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P e951f009258e473cb57cf8773c7975d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.976104 21710 tablet_bootstrap.cc:654] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:42.976819 21717 tablet_bootstrap.cc:654] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:42.977237 21712 tablet_bootstrap.cc:492] T 880876d792554606a7ccf21f9ec08e0f P b7604c0eb3bf4a7cb260b564a94a1191: No bootstrap required, opened a new log
I20240502 07:59:42.977242 21707 ts_tablet_manager.cc:1398] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P e951f009258e473cb57cf8773c7975d4: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:42.977317 21712 ts_tablet_manager.cc:1367] T 880876d792554606a7ccf21f9ec08e0f P b7604c0eb3bf4a7cb260b564a94a1191: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.000s
I20240502 07:59:42.977888 21712 raft_consensus.cc:348] T 880876d792554606a7ccf21f9ec08e0f P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.977986 21712 raft_consensus.cc:374] T 880876d792554606a7ccf21f9ec08e0f P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:42.978014 21712 raft_consensus.cc:729] T 880876d792554606a7ccf21f9ec08e0f P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7604c0eb3bf4a7cb260b564a94a1191, State: Initialized, Role: FOLLOWER
I20240502 07:59:42.978157 21712 consensus_queue.cc:260] T 880876d792554606a7ccf21f9ec08e0f P b7604c0eb3bf4a7cb260b564a94a1191 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.978518 21712 ts_tablet_manager.cc:1398] T 880876d792554606a7ccf21f9ec08e0f P b7604c0eb3bf4a7cb260b564a94a1191: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:42.978725 21710 tablet_bootstrap.cc:492] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07: No bootstrap required, opened a new log
I20240502 07:59:42.978736 21712 tablet_bootstrap.cc:492] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b7604c0eb3bf4a7cb260b564a94a1191: Bootstrap starting.
I20240502 07:59:42.978811 21710 ts_tablet_manager.cc:1367] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 07:59:42.979559 21712 tablet_bootstrap.cc:654] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b7604c0eb3bf4a7cb260b564a94a1191: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:42.979554 21717 tablet_bootstrap.cc:492] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8: No bootstrap required, opened a new log
I20240502 07:59:42.979660 21717 ts_tablet_manager.cc:1367] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:59:42.980113 21707 tablet_bootstrap.cc:492] T 880876d792554606a7ccf21f9ec08e0f P e951f009258e473cb57cf8773c7975d4: Bootstrap starting.
I20240502 07:59:42.980084 21717 raft_consensus.cc:348] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.980268 21717 raft_consensus.cc:374] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:42.980310 21717 raft_consensus.cc:729] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5f471f8b5944986a7d878cf6ac550b8, State: Initialized, Role: FOLLOWER
I20240502 07:59:42.980537 21710 raft_consensus.cc:348] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.980651 21710 raft_consensus.cc:374] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:42.980681 21710 raft_consensus.cc:729] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35642aa2099c40e59bc2092604078e07, State: Initialized, Role: FOLLOWER
I20240502 07:59:42.980824 21717 consensus_queue.cc:260] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.980971 21712 tablet_bootstrap.cc:492] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b7604c0eb3bf4a7cb260b564a94a1191: No bootstrap required, opened a new log
I20240502 07:59:42.981050 21712 ts_tablet_manager.cc:1367] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b7604c0eb3bf4a7cb260b564a94a1191: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 07:59:42.981030 21710 consensus_queue.cc:260] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.981500 21712 raft_consensus.cc:348] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.981588 21707 tablet_bootstrap.cc:654] T 880876d792554606a7ccf21f9ec08e0f P e951f009258e473cb57cf8773c7975d4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:42.981609 21712 raft_consensus.cc:374] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:42.981640 21712 raft_consensus.cc:729] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7604c0eb3bf4a7cb260b564a94a1191, State: Initialized, Role: FOLLOWER
I20240502 07:59:42.981789 21717 ts_tablet_manager.cc:1398] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:42.981752 21712 consensus_queue.cc:260] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b7604c0eb3bf4a7cb260b564a94a1191 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.981952 21717 tablet_bootstrap.cc:492] T 880876d792554606a7ccf21f9ec08e0f P b5f471f8b5944986a7d878cf6ac550b8: Bootstrap starting.
I20240502 07:59:42.982324 21712 ts_tablet_manager.cc:1398] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b7604c0eb3bf4a7cb260b564a94a1191: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:42.982764 21717 tablet_bootstrap.cc:654] T 880876d792554606a7ccf21f9ec08e0f P b5f471f8b5944986a7d878cf6ac550b8: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:42.983492 21710 ts_tablet_manager.cc:1398] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20240502 07:59:42.984145 21710 tablet_bootstrap.cc:492] T 880876d792554606a7ccf21f9ec08e0f P 35642aa2099c40e59bc2092604078e07: Bootstrap starting.
I20240502 07:59:42.984338 21709 tablet_bootstrap.cc:492] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 29a7180fe4574c09a8dce7c27500aab2: No bootstrap required, opened a new log
I20240502 07:59:42.984428 21709 ts_tablet_manager.cc:1367] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 29a7180fe4574c09a8dce7c27500aab2: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.000s
I20240502 07:59:42.984529 21708 tablet_bootstrap.cc:492] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P c15e23a34059456788f9775031e27ec0: No bootstrap required, opened a new log
I20240502 07:59:42.984598 21708 ts_tablet_manager.cc:1367] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P c15e23a34059456788f9775031e27ec0: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20240502 07:59:42.984851 21709 raft_consensus.cc:348] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.984966 21709 raft_consensus.cc:374] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:42.984997 21709 raft_consensus.cc:729] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29a7180fe4574c09a8dce7c27500aab2, State: Initialized, Role: FOLLOWER
I20240502 07:59:42.985036 21710 tablet_bootstrap.cc:654] T 880876d792554606a7ccf21f9ec08e0f P 35642aa2099c40e59bc2092604078e07: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:42.984977 21708 raft_consensus.cc:348] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.985107 21708 raft_consensus.cc:374] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:42.985139 21708 raft_consensus.cc:729] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c15e23a34059456788f9775031e27ec0, State: Initialized, Role: FOLLOWER
I20240502 07:59:42.985116 21709 consensus_queue.cc:260] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 29a7180fe4574c09a8dce7c27500aab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.985246 21708 consensus_queue.cc:260] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P c15e23a34059456788f9775031e27ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.985391 21709 ts_tablet_manager.cc:1398] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 29a7180fe4574c09a8dce7c27500aab2: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:42.985509 21708 ts_tablet_manager.cc:1398] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P c15e23a34059456788f9775031e27ec0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:42.987282 21707 tablet_bootstrap.cc:492] T 880876d792554606a7ccf21f9ec08e0f P e951f009258e473cb57cf8773c7975d4: No bootstrap required, opened a new log
I20240502 07:59:42.987361 21707 ts_tablet_manager.cc:1367] T 880876d792554606a7ccf21f9ec08e0f P e951f009258e473cb57cf8773c7975d4: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240502 07:59:42.987387 21710 tablet_bootstrap.cc:492] T 880876d792554606a7ccf21f9ec08e0f P 35642aa2099c40e59bc2092604078e07: No bootstrap required, opened a new log
I20240502 07:59:42.987447 21710 ts_tablet_manager.cc:1367] T 880876d792554606a7ccf21f9ec08e0f P 35642aa2099c40e59bc2092604078e07: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:59:42.987452 21717 tablet_bootstrap.cc:492] T 880876d792554606a7ccf21f9ec08e0f P b5f471f8b5944986a7d878cf6ac550b8: No bootstrap required, opened a new log
I20240502 07:59:42.987524 21717 ts_tablet_manager.cc:1367] T 880876d792554606a7ccf21f9ec08e0f P b5f471f8b5944986a7d878cf6ac550b8: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240502 07:59:42.987780 21710 raft_consensus.cc:348] T 880876d792554606a7ccf21f9ec08e0f P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.987768 21707 raft_consensus.cc:348] T 880876d792554606a7ccf21f9ec08e0f P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.987874 21710 raft_consensus.cc:374] T 880876d792554606a7ccf21f9ec08e0f P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:42.987874 21707 raft_consensus.cc:374] T 880876d792554606a7ccf21f9ec08e0f P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:42.987903 21710 raft_consensus.cc:729] T 880876d792554606a7ccf21f9ec08e0f P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35642aa2099c40e59bc2092604078e07, State: Initialized, Role: FOLLOWER
I20240502 07:59:42.987903 21707 raft_consensus.cc:729] T 880876d792554606a7ccf21f9ec08e0f P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e951f009258e473cb57cf8773c7975d4, State: Initialized, Role: FOLLOWER
I20240502 07:59:42.987919 21717 raft_consensus.cc:348] T 880876d792554606a7ccf21f9ec08e0f P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.988025 21717 raft_consensus.cc:374] T 880876d792554606a7ccf21f9ec08e0f P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:42.988057 21717 raft_consensus.cc:729] T 880876d792554606a7ccf21f9ec08e0f P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5f471f8b5944986a7d878cf6ac550b8, State: Initialized, Role: FOLLOWER
I20240502 07:59:42.988011 21707 consensus_queue.cc:260] T 880876d792554606a7ccf21f9ec08e0f P e951f009258e473cb57cf8773c7975d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.988010 21710 consensus_queue.cc:260] T 880876d792554606a7ccf21f9ec08e0f P 35642aa2099c40e59bc2092604078e07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.988166 21717 consensus_queue.cc:260] T 880876d792554606a7ccf21f9ec08e0f P b5f471f8b5944986a7d878cf6ac550b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:42.988258 21707 ts_tablet_manager.cc:1398] T 880876d792554606a7ccf21f9ec08e0f P e951f009258e473cb57cf8773c7975d4: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:42.988279 21710 ts_tablet_manager.cc:1398] T 880876d792554606a7ccf21f9ec08e0f P 35642aa2099c40e59bc2092604078e07: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:42.988440 21717 ts_tablet_manager.cc:1398] T 880876d792554606a7ccf21f9ec08e0f P b5f471f8b5944986a7d878cf6ac550b8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:43.037029 21464 raft_consensus.cc:482] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:43.037163 21464 raft_consensus.cc:504] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:43.038051 20669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbf4b90cac9e4ff3b2936a1fa2ca8561" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b443e68b6212450286869124684bdfff" is_pre_election: true
I20240502 07:59:43.038058 20801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbf4b90cac9e4ff3b2936a1fa2ca8561" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15e23a34059456788f9775031e27ec0" is_pre_election: true
I20240502 07:59:43.038168 21464 leader_election.cc:288] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b443e68b6212450286869124684bdfff (127.3.139.69:37923), c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085), b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577), 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703), b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331), 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587)
I20240502 07:59:43.038203 20669 raft_consensus.cc:2454] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e951f009258e473cb57cf8773c7975d4 in term 0.
I20240502 07:59:43.038224 20801 raft_consensus.cc:2454] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e951f009258e473cb57cf8773c7975d4 in term 0.
I20240502 07:59:43.038575 20275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbf4b90cac9e4ff3b2936a1fa2ca8561" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" is_pre_election: true
I20240502 07:59:43.038589 20935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbf4b90cac9e4ff3b2936a1fa2ca8561" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f471f8b5944986a7d878cf6ac550b8" is_pre_election: true
I20240502 07:59:43.038659 20407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbf4b90cac9e4ff3b2936a1fa2ca8561" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35642aa2099c40e59bc2092604078e07" is_pre_election: true
I20240502 07:59:43.038702 20275 raft_consensus.cc:2454] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e951f009258e473cb57cf8773c7975d4 in term 0.
I20240502 07:59:43.038710 20935 raft_consensus.cc:2454] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e951f009258e473cb57cf8773c7975d4 in term 0.
I20240502 07:59:43.038765 20407 raft_consensus.cc:2454] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e951f009258e473cb57cf8773c7975d4 in term 0.
I20240502 07:59:43.038740 20142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbf4b90cac9e4ff3b2936a1fa2ca8561" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7180fe4574c09a8dce7c27500aab2" is_pre_election: true
I20240502 07:59:43.038853 20142 raft_consensus.cc:2454] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e951f009258e473cb57cf8773c7975d4 in term 0.
I20240502 07:59:43.039031 20475 leader_election.cc:302] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 35642aa2099c40e59bc2092604078e07, b7604c0eb3bf4a7cb260b564a94a1191, c15e23a34059456788f9775031e27ec0, e951f009258e473cb57cf8773c7975d4; no voters:
I20240502 07:59:43.039242 21464 raft_consensus.cc:2781] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:43.039302 21464 raft_consensus.cc:482] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:43.039330 21464 raft_consensus.cc:3037] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:43.040169 21464 raft_consensus.cc:504] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:43.040696 21464 leader_election.cc:288] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [CANDIDATE]: Term 1 election: Requested vote from peers b443e68b6212450286869124684bdfff (127.3.139.69:37923), c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085), b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577), 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703), b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331), 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587)
I20240502 07:59:43.040944 20669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbf4b90cac9e4ff3b2936a1fa2ca8561" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b443e68b6212450286869124684bdfff"
I20240502 07:59:43.040985 20407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbf4b90cac9e4ff3b2936a1fa2ca8561" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35642aa2099c40e59bc2092604078e07"
I20240502 07:59:43.041069 20407 raft_consensus.cc:3037] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:43.041069 20669 raft_consensus.cc:3037] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:43.041082 20935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbf4b90cac9e4ff3b2936a1fa2ca8561" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f471f8b5944986a7d878cf6ac550b8"
I20240502 07:59:43.041172 20142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbf4b90cac9e4ff3b2936a1fa2ca8561" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7180fe4574c09a8dce7c27500aab2"
I20240502 07:59:43.041198 20935 raft_consensus.cc:3037] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:43.041255 20142 raft_consensus.cc:3037] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:43.041980 20407 raft_consensus.cc:2454] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e951f009258e473cb57cf8773c7975d4 in term 1.
I20240502 07:59:43.041986 20669 raft_consensus.cc:2454] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e951f009258e473cb57cf8773c7975d4 in term 1.
I20240502 07:59:43.042096 20935 raft_consensus.cc:2454] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e951f009258e473cb57cf8773c7975d4 in term 1.
I20240502 07:59:43.042117 20142 raft_consensus.cc:2454] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e951f009258e473cb57cf8773c7975d4 in term 1.
I20240502 07:59:43.042214 20801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbf4b90cac9e4ff3b2936a1fa2ca8561" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15e23a34059456788f9775031e27ec0"
I20240502 07:59:43.042313 20801 raft_consensus.cc:3037] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:43.042315 20275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbf4b90cac9e4ff3b2936a1fa2ca8561" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7604c0eb3bf4a7cb260b564a94a1191"
I20240502 07:59:43.042410 20275 raft_consensus.cc:3037] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:43.042831 20473 leader_election.cc:302] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 29a7180fe4574c09a8dce7c27500aab2, 35642aa2099c40e59bc2092604078e07, b443e68b6212450286869124684bdfff, e951f009258e473cb57cf8773c7975d4; no voters:
I20240502 07:59:43.043025 21464 raft_consensus.cc:2781] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:43.043133 21464 raft_consensus.cc:686] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [term 1 LEADER]: Becoming Leader. State: Replica: e951f009258e473cb57cf8773c7975d4, State: Running, Role: LEADER
I20240502 07:59:43.043283 20801 raft_consensus.cc:2454] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e951f009258e473cb57cf8773c7975d4 in term 1.
I20240502 07:59:43.043282 21464 consensus_queue.cc:237] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:43.043391 20275 raft_consensus.cc:2454] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e951f009258e473cb57cf8773c7975d4 in term 1.
I20240502 07:59:43.043603 21428 raft_consensus.cc:482] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:43.043690 21428 raft_consensus.cc:504] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:43.044328 21428 leader_election.cc:288] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331), 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587), b443e68b6212450286869124684bdfff (127.3.139.69:37923), e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315), c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085), b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577)
I20240502 07:59:43.044417 20407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "880876d792554606a7ccf21f9ec08e0f" candidate_uuid: "29a7180fe4574c09a8dce7c27500aab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35642aa2099c40e59bc2092604078e07" is_pre_election: true
I20240502 07:59:43.044518 20407 raft_consensus.cc:2454] T 880876d792554606a7ccf21f9ec08e0f P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29a7180fe4574c09a8dce7c27500aab2 in term 0.
I20240502 07:59:43.044955 20935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "880876d792554606a7ccf21f9ec08e0f" candidate_uuid: "29a7180fe4574c09a8dce7c27500aab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f471f8b5944986a7d878cf6ac550b8" is_pre_election: true
I20240502 07:59:43.044955 20801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "880876d792554606a7ccf21f9ec08e0f" candidate_uuid: "29a7180fe4574c09a8dce7c27500aab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15e23a34059456788f9775031e27ec0" is_pre_election: true
I20240502 07:59:43.045050 20801 raft_consensus.cc:2454] T 880876d792554606a7ccf21f9ec08e0f P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29a7180fe4574c09a8dce7c27500aab2 in term 0.
I20240502 07:59:43.045050 20935 raft_consensus.cc:2454] T 880876d792554606a7ccf21f9ec08e0f P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29a7180fe4574c09a8dce7c27500aab2 in term 0.
I20240502 07:59:43.045207 20669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "880876d792554606a7ccf21f9ec08e0f" candidate_uuid: "29a7180fe4574c09a8dce7c27500aab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b443e68b6212450286869124684bdfff" is_pre_election: true
I20240502 07:59:43.045316 20669 raft_consensus.cc:2454] T 880876d792554606a7ccf21f9ec08e0f P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29a7180fe4574c09a8dce7c27500aab2 in term 0.
I20240502 07:59:43.045375 20275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "880876d792554606a7ccf21f9ec08e0f" candidate_uuid: "29a7180fe4574c09a8dce7c27500aab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" is_pre_election: true
I20240502 07:59:43.045397 20539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "880876d792554606a7ccf21f9ec08e0f" candidate_uuid: "29a7180fe4574c09a8dce7c27500aab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e951f009258e473cb57cf8773c7975d4" is_pre_election: true
I20240502 07:59:43.045471 20275 raft_consensus.cc:2454] T 880876d792554606a7ccf21f9ec08e0f P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29a7180fe4574c09a8dce7c27500aab2 in term 0.
I20240502 07:59:43.045478 20539 raft_consensus.cc:2454] T 880876d792554606a7ccf21f9ec08e0f P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29a7180fe4574c09a8dce7c27500aab2 in term 0.
I20240502 07:59:43.045680 20077 leader_election.cc:302] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 29a7180fe4574c09a8dce7c27500aab2, 35642aa2099c40e59bc2092604078e07, b443e68b6212450286869124684bdfff, c15e23a34059456788f9775031e27ec0; no voters:
I20240502 07:59:43.045568 20001 catalog_manager.cc:5484] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to e951f009258e473cb57cf8773c7975d4 (127.3.139.68). New cstate: current_term: 1 leader_uuid: "e951f009258e473cb57cf8773c7975d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:43.045917 21428 raft_consensus.cc:2781] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:43.045974 21428 raft_consensus.cc:482] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:43.046018 21428 raft_consensus.cc:3037] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:43.046663 21428 raft_consensus.cc:504] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:43.047178 21428 leader_election.cc:288] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [CANDIDATE]: Term 1 election: Requested vote from peers b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331), 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587), b443e68b6212450286869124684bdfff (127.3.139.69:37923), e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315), c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085), b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577)
I20240502 07:59:43.047470 20669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "880876d792554606a7ccf21f9ec08e0f" candidate_uuid: "29a7180fe4574c09a8dce7c27500aab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b443e68b6212450286869124684bdfff"
I20240502 07:59:43.047528 20407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "880876d792554606a7ccf21f9ec08e0f" candidate_uuid: "29a7180fe4574c09a8dce7c27500aab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35642aa2099c40e59bc2092604078e07"
I20240502 07:59:43.047570 20669 raft_consensus.cc:3037] T 880876d792554606a7ccf21f9ec08e0f P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:43.047610 20407 raft_consensus.cc:3037] T 880876d792554606a7ccf21f9ec08e0f P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:43.047655 20539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "880876d792554606a7ccf21f9ec08e0f" candidate_uuid: "29a7180fe4574c09a8dce7c27500aab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e951f009258e473cb57cf8773c7975d4"
I20240502 07:59:43.047755 20539 raft_consensus.cc:3037] T 880876d792554606a7ccf21f9ec08e0f P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:43.047987 20801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "880876d792554606a7ccf21f9ec08e0f" candidate_uuid: "29a7180fe4574c09a8dce7c27500aab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15e23a34059456788f9775031e27ec0"
I20240502 07:59:43.048069 20801 raft_consensus.cc:3037] T 880876d792554606a7ccf21f9ec08e0f P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:43.048377 20407 raft_consensus.cc:2454] T 880876d792554606a7ccf21f9ec08e0f P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29a7180fe4574c09a8dce7c27500aab2 in term 1.
I20240502 07:59:43.048377 20669 raft_consensus.cc:2454] T 880876d792554606a7ccf21f9ec08e0f P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29a7180fe4574c09a8dce7c27500aab2 in term 1.
I20240502 07:59:43.048620 20539 raft_consensus.cc:2454] T 880876d792554606a7ccf21f9ec08e0f P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29a7180fe4574c09a8dce7c27500aab2 in term 1.
I20240502 07:59:43.048781 20801 raft_consensus.cc:2454] T 880876d792554606a7ccf21f9ec08e0f P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29a7180fe4574c09a8dce7c27500aab2 in term 1.
I20240502 07:59:43.048861 20275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "880876d792554606a7ccf21f9ec08e0f" candidate_uuid: "29a7180fe4574c09a8dce7c27500aab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7604c0eb3bf4a7cb260b564a94a1191"
I20240502 07:59:43.048903 20077 leader_election.cc:302] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 29a7180fe4574c09a8dce7c27500aab2, 35642aa2099c40e59bc2092604078e07, b443e68b6212450286869124684bdfff, e951f009258e473cb57cf8773c7975d4; no voters:
I20240502 07:59:43.048952 20275 raft_consensus.cc:3037] T 880876d792554606a7ccf21f9ec08e0f P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:43.048995 20935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "880876d792554606a7ccf21f9ec08e0f" candidate_uuid: "29a7180fe4574c09a8dce7c27500aab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f471f8b5944986a7d878cf6ac550b8"
I20240502 07:59:43.049084 20935 raft_consensus.cc:3037] T 880876d792554606a7ccf21f9ec08e0f P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:43.049109 21428 raft_consensus.cc:2781] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:43.049304 21428 raft_consensus.cc:686] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [term 1 LEADER]: Becoming Leader. State: Replica: 29a7180fe4574c09a8dce7c27500aab2, State: Running, Role: LEADER
I20240502 07:59:43.049629 21428 consensus_queue.cc:237] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:43.049715 20275 raft_consensus.cc:2454] T 880876d792554606a7ccf21f9ec08e0f P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29a7180fe4574c09a8dce7c27500aab2 in term 1.
I20240502 07:59:43.049871 20935 raft_consensus.cc:2454] T 880876d792554606a7ccf21f9ec08e0f P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29a7180fe4574c09a8dce7c27500aab2 in term 1.
I20240502 07:59:43.052251 20001 catalog_manager.cc:5484] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65). New cstate: current_term: 1 leader_uuid: "29a7180fe4574c09a8dce7c27500aab2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:43.212436 21706 raft_consensus.cc:482] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:43.212554 21706 raft_consensus.cc:504] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:43.213246 21706 leader_election.cc:288] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331), 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703), 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587), b443e68b6212450286869124684bdfff (127.3.139.69:37923), e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315), c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085)
I20240502 07:59:43.213461 20142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f588a3ec49a24b1a9a9e19cdf8c30ca5" candidate_uuid: "b5f471f8b5944986a7d878cf6ac550b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7180fe4574c09a8dce7c27500aab2" is_pre_election: true
I20240502 07:59:43.213531 20407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f588a3ec49a24b1a9a9e19cdf8c30ca5" candidate_uuid: "b5f471f8b5944986a7d878cf6ac550b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35642aa2099c40e59bc2092604078e07" is_pre_election: true
I20240502 07:59:43.213608 20142 raft_consensus.cc:2454] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5f471f8b5944986a7d878cf6ac550b8 in term 0.
I20240502 07:59:43.213665 20407 raft_consensus.cc:2454] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5f471f8b5944986a7d878cf6ac550b8 in term 0.
I20240502 07:59:43.213928 20539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f588a3ec49a24b1a9a9e19cdf8c30ca5" candidate_uuid: "b5f471f8b5944986a7d878cf6ac550b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e951f009258e473cb57cf8773c7975d4" is_pre_election: true
I20240502 07:59:43.213950 20801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f588a3ec49a24b1a9a9e19cdf8c30ca5" candidate_uuid: "b5f471f8b5944986a7d878cf6ac550b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15e23a34059456788f9775031e27ec0" is_pre_election: true
I20240502 07:59:43.214062 20801 raft_consensus.cc:2454] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5f471f8b5944986a7d878cf6ac550b8 in term 0.
I20240502 07:59:43.214061 20539 raft_consensus.cc:2454] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5f471f8b5944986a7d878cf6ac550b8 in term 0.
I20240502 07:59:43.214419 20669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f588a3ec49a24b1a9a9e19cdf8c30ca5" candidate_uuid: "b5f471f8b5944986a7d878cf6ac550b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b443e68b6212450286869124684bdfff" is_pre_election: true
I20240502 07:59:43.214547 20669 raft_consensus.cc:2454] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5f471f8b5944986a7d878cf6ac550b8 in term 0.
I20240502 07:59:43.214581 20275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f588a3ec49a24b1a9a9e19cdf8c30ca5" candidate_uuid: "b5f471f8b5944986a7d878cf6ac550b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" is_pre_election: true
I20240502 07:59:43.214603 20869 leader_election.cc:302] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 29a7180fe4574c09a8dce7c27500aab2, 35642aa2099c40e59bc2092604078e07, b5f471f8b5944986a7d878cf6ac550b8, e951f009258e473cb57cf8773c7975d4; no voters:
I20240502 07:59:43.214704 20275 raft_consensus.cc:2454] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5f471f8b5944986a7d878cf6ac550b8 in term 0.
I20240502 07:59:43.214900 21706 raft_consensus.cc:2781] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:43.215000 21706 raft_consensus.cc:482] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:43.215040 21706 raft_consensus.cc:3037] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:43.216218 21706 raft_consensus.cc:504] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:43.216723 21706 leader_election.cc:288] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [CANDIDATE]: Term 1 election: Requested vote from peers b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331), 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703), 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587), b443e68b6212450286869124684bdfff (127.3.139.69:37923), e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315), c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085)
I20240502 07:59:43.216972 20407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f588a3ec49a24b1a9a9e19cdf8c30ca5" candidate_uuid: "b5f471f8b5944986a7d878cf6ac550b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35642aa2099c40e59bc2092604078e07"
I20240502 07:59:43.217028 20801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f588a3ec49a24b1a9a9e19cdf8c30ca5" candidate_uuid: "b5f471f8b5944986a7d878cf6ac550b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15e23a34059456788f9775031e27ec0"
I20240502 07:59:43.217084 20407 raft_consensus.cc:3037] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:43.217133 20801 raft_consensus.cc:3037] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:43.217164 20142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f588a3ec49a24b1a9a9e19cdf8c30ca5" candidate_uuid: "b5f471f8b5944986a7d878cf6ac550b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7180fe4574c09a8dce7c27500aab2"
I20240502 07:59:43.217257 20142 raft_consensus.cc:3037] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:43.217305 20275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f588a3ec49a24b1a9a9e19cdf8c30ca5" candidate_uuid: "b5f471f8b5944986a7d878cf6ac550b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7604c0eb3bf4a7cb260b564a94a1191"
I20240502 07:59:43.217401 20275 raft_consensus.cc:3037] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:43.218008 20407 raft_consensus.cc:2454] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5f471f8b5944986a7d878cf6ac550b8 in term 1.
I20240502 07:59:43.218027 20801 raft_consensus.cc:2454] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5f471f8b5944986a7d878cf6ac550b8 in term 1.
I20240502 07:59:43.218119 20142 raft_consensus.cc:2454] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5f471f8b5944986a7d878cf6ac550b8 in term 1.
I20240502 07:59:43.218240 20669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f588a3ec49a24b1a9a9e19cdf8c30ca5" candidate_uuid: "b5f471f8b5944986a7d878cf6ac550b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b443e68b6212450286869124684bdfff"
I20240502 07:59:43.218338 20669 raft_consensus.cc:3037] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:43.218519 20275 raft_consensus.cc:2454] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5f471f8b5944986a7d878cf6ac550b8 in term 1.
I20240502 07:59:43.218619 20539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f588a3ec49a24b1a9a9e19cdf8c30ca5" candidate_uuid: "b5f471f8b5944986a7d878cf6ac550b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e951f009258e473cb57cf8773c7975d4"
I20240502 07:59:43.218732 20539 raft_consensus.cc:3037] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:43.218819 20870 leader_election.cc:302] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 29a7180fe4574c09a8dce7c27500aab2, b5f471f8b5944986a7d878cf6ac550b8, b7604c0eb3bf4a7cb260b564a94a1191, c15e23a34059456788f9775031e27ec0; no voters:
I20240502 07:59:43.218979 21706 raft_consensus.cc:2781] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:43.219076 21706 raft_consensus.cc:686] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 LEADER]: Becoming Leader. State: Replica: b5f471f8b5944986a7d878cf6ac550b8, State: Running, Role: LEADER
I20240502 07:59:43.219213 20669 raft_consensus.cc:2454] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5f471f8b5944986a7d878cf6ac550b8 in term 1.
I20240502 07:59:43.219223 21706 consensus_queue.cc:237] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:43.219650 20539 raft_consensus.cc:2454] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5f471f8b5944986a7d878cf6ac550b8 in term 1.
I20240502 07:59:43.221331 20001 catalog_manager.cc:5484] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 reported cstate change: term changed from 0 to 1, leader changed from <none> to b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71). New cstate: current_term: 1 leader_uuid: "b5f471f8b5944986a7d878cf6ac550b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: HEALTHY } } }
W20240502 07:59:43.347824 21739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:43.348047 21739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:43.350950 21739 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_2'
I20240502 07:59:43.422891 21464 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:43.431082 20142 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:43.431495 21464 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:43.431597 20669 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:43.432252 20935 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:43.432878 21545 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:43.433125 21545 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:43.435024 20407 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:43.435451 21706 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:43.436122 20275 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:43.436619 20801 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:43.436666 21706 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:43.437510 20539 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:43.437813 20406 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 07:59:43.439965 20669 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:43.440346 21676 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:43.440970 20801 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 07:59:43.441298 21785 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:43.443277 21786 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:43.443809 20142 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:43.443008 21706 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240502 07:59:43.444175 21706 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:43.447168 20802 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:43.447713 21736 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:43.449331 20405 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:43.449844 20273 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:43.450229 21428 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:43.450510 21428 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:43.450564 20934 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:43.450718 20670 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:43.450733 21676 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:43.451443 20538 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:43.452028 21428 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:43.452303 21428 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:43.458501 21796 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 197.363 ms
time per row: 0.0986817 ms
I20240502 07:59:43.592983 19999 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33518:
name: "rebalance_test_table_3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 7
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240502 07:59:43.593544 19999 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_3 in case of a server failure: 8 tablet servers would be needed, 7 are available. Consider bringing up more tablet servers.
I20240502 07:59:43.601655 20518 tablet_service.cc:1467] Processing CreateTablet for tablet bbae35bae8c047a6982e128ffb77ca86 (DEFAULT_TABLE table=rebalance_test_table_3 [id=1388635dd26f4d3791ee92dfbf8d374a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:43.601989 20518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbae35bae8c047a6982e128ffb77ca86. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:43.603188 20519 tablet_service.cc:1467] Processing CreateTablet for tablet e61fe3cb61c240f9b2c898aff1951144 (DEFAULT_TABLE table=rebalance_test_table_3 [id=1388635dd26f4d3791ee92dfbf8d374a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:43.603418 20519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e61fe3cb61c240f9b2c898aff1951144. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:43.605564 20651 tablet_service.cc:1467] Processing CreateTablet for tablet e61fe3cb61c240f9b2c898aff1951144 (DEFAULT_TABLE table=rebalance_test_table_3 [id=1388635dd26f4d3791ee92dfbf8d374a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:43.605821 20651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e61fe3cb61c240f9b2c898aff1951144. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:43.607389 20386 tablet_service.cc:1467] Processing CreateTablet for tablet c3ca6f24c92d4273ac6c9d83161813d7 (DEFAULT_TABLE table=rebalance_test_table_3 [id=1388635dd26f4d3791ee92dfbf8d374a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:43.607424 20650 tablet_service.cc:1467] Processing CreateTablet for tablet bbae35bae8c047a6982e128ffb77ca86 (DEFAULT_TABLE table=rebalance_test_table_3 [id=1388635dd26f4d3791ee92dfbf8d374a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:43.607623 20386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3ca6f24c92d4273ac6c9d83161813d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:43.607622 20650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbae35bae8c047a6982e128ffb77ca86. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:43.607846 20649 tablet_service.cc:1467] Processing CreateTablet for tablet c3ca6f24c92d4273ac6c9d83161813d7 (DEFAULT_TABLE table=rebalance_test_table_3 [id=1388635dd26f4d3791ee92dfbf8d374a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:43.608047 20649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3ca6f24c92d4273ac6c9d83161813d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:43.607632 21852 tablet_bootstrap.cc:492] T e61fe3cb61c240f9b2c898aff1951144 P b443e68b6212450286869124684bdfff: Bootstrap starting.
I20240502 07:59:43.609952 21852 tablet_bootstrap.cc:654] T e61fe3cb61c240f9b2c898aff1951144 P b443e68b6212450286869124684bdfff: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:43.610056 20253 tablet_service.cc:1467] Processing CreateTablet for tablet e61fe3cb61c240f9b2c898aff1951144 (DEFAULT_TABLE table=rebalance_test_table_3 [id=1388635dd26f4d3791ee92dfbf8d374a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:43.610306 20253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e61fe3cb61c240f9b2c898aff1951144. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:43.610658 21853 tablet_bootstrap.cc:492] T c3ca6f24c92d4273ac6c9d83161813d7 P 35642aa2099c40e59bc2092604078e07: Bootstrap starting.
I20240502 07:59:43.611037 21852 tablet_bootstrap.cc:492] T e61fe3cb61c240f9b2c898aff1951144 P b443e68b6212450286869124684bdfff: No bootstrap required, opened a new log
I20240502 07:59:43.611150 21852 ts_tablet_manager.cc:1367] T e61fe3cb61c240f9b2c898aff1951144 P b443e68b6212450286869124684bdfff: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:59:43.611369 20387 tablet_service.cc:1467] Processing CreateTablet for tablet bbae35bae8c047a6982e128ffb77ca86 (DEFAULT_TABLE table=rebalance_test_table_3 [id=1388635dd26f4d3791ee92dfbf8d374a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:43.611567 20387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbae35bae8c047a6982e128ffb77ca86. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:43.611557 21852 raft_consensus.cc:348] T e61fe3cb61c240f9b2c898aff1951144 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:43.611665 21852 raft_consensus.cc:374] T e61fe3cb61c240f9b2c898aff1951144 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:43.611692 21852 raft_consensus.cc:729] T e61fe3cb61c240f9b2c898aff1951144 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b443e68b6212450286869124684bdfff, State: Initialized, Role: FOLLOWER
I20240502 07:59:43.611761 20385 tablet_service.cc:1467] Processing CreateTablet for tablet e61fe3cb61c240f9b2c898aff1951144 (DEFAULT_TABLE table=rebalance_test_table_3 [id=1388635dd26f4d3791ee92dfbf8d374a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:43.611828 21852 consensus_queue.cc:260] T e61fe3cb61c240f9b2c898aff1951144 P b443e68b6212450286869124684bdfff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:43.611958 20385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e61fe3cb61c240f9b2c898aff1951144. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:43.612182 21852 ts_tablet_manager.cc:1398] T e61fe3cb61c240f9b2c898aff1951144 P b443e68b6212450286869124684bdfff: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:43.612526 20255 tablet_service.cc:1467] Processing CreateTablet for tablet bbae35bae8c047a6982e128ffb77ca86 (DEFAULT_TABLE table=rebalance_test_table_3 [id=1388635dd26f4d3791ee92dfbf8d374a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:43.612577 21852 tablet_bootstrap.cc:492] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff: Bootstrap starting.
I20240502 07:59:43.612733 20255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbae35bae8c047a6982e128ffb77ca86. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:43.613514 21852 tablet_bootstrap.cc:654] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:43.611557 21853 tablet_bootstrap.cc:654] T c3ca6f24c92d4273ac6c9d83161813d7 P 35642aa2099c40e59bc2092604078e07: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:43.614109 21855 tablet_bootstrap.cc:492] T e61fe3cb61c240f9b2c898aff1951144 P b7604c0eb3bf4a7cb260b564a94a1191: Bootstrap starting.
I20240502 07:59:43.614598 20254 tablet_service.cc:1467] Processing CreateTablet for tablet c3ca6f24c92d4273ac6c9d83161813d7 (DEFAULT_TABLE table=rebalance_test_table_3 [id=1388635dd26f4d3791ee92dfbf8d374a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:43.614813 20254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3ca6f24c92d4273ac6c9d83161813d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:43.614810 21855 tablet_bootstrap.cc:654] T e61fe3cb61c240f9b2c898aff1951144 P b7604c0eb3bf4a7cb260b564a94a1191: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:43.615372 21853 tablet_bootstrap.cc:492] T c3ca6f24c92d4273ac6c9d83161813d7 P 35642aa2099c40e59bc2092604078e07: No bootstrap required, opened a new log
I20240502 07:59:43.615449 21853 ts_tablet_manager.cc:1367] T c3ca6f24c92d4273ac6c9d83161813d7 P 35642aa2099c40e59bc2092604078e07: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 07:59:43.615782 21853 raft_consensus.cc:348] T c3ca6f24c92d4273ac6c9d83161813d7 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:43.615885 21853 raft_consensus.cc:374] T c3ca6f24c92d4273ac6c9d83161813d7 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:43.615908 21853 raft_consensus.cc:729] T c3ca6f24c92d4273ac6c9d83161813d7 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35642aa2099c40e59bc2092604078e07, State: Initialized, Role: FOLLOWER
I20240502 07:59:43.616006 21853 consensus_queue.cc:260] T c3ca6f24c92d4273ac6c9d83161813d7 P 35642aa2099c40e59bc2092604078e07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:43.616461 21852 tablet_bootstrap.cc:492] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff: No bootstrap required, opened a new log
I20240502 07:59:43.616531 21852 ts_tablet_manager.cc:1367] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240502 07:59:43.616616 21853 ts_tablet_manager.cc:1398] T c3ca6f24c92d4273ac6c9d83161813d7 P 35642aa2099c40e59bc2092604078e07: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:43.616768 21853 tablet_bootstrap.cc:492] T bbae35bae8c047a6982e128ffb77ca86 P 35642aa2099c40e59bc2092604078e07: Bootstrap starting.
I20240502 07:59:43.616825 20783 tablet_service.cc:1467] Processing CreateTablet for tablet e61fe3cb61c240f9b2c898aff1951144 (DEFAULT_TABLE table=rebalance_test_table_3 [id=1388635dd26f4d3791ee92dfbf8d374a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:43.616961 21852 raft_consensus.cc:348] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:43.617061 21852 raft_consensus.cc:374] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:43.617089 20783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e61fe3cb61c240f9b2c898aff1951144. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:43.617089 21852 raft_consensus.cc:729] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b443e68b6212450286869124684bdfff, State: Initialized, Role: FOLLOWER
I20240502 07:59:43.617193 21852 consensus_queue.cc:260] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:43.617470 21852 ts_tablet_manager.cc:1398] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:43.617605 21852 tablet_bootstrap.cc:492] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff: Bootstrap starting.
I20240502 07:59:43.617765 21853 tablet_bootstrap.cc:654] T bbae35bae8c047a6982e128ffb77ca86 P 35642aa2099c40e59bc2092604078e07: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:43.618353 21852 tablet_bootstrap.cc:654] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:43.618708 21853 tablet_bootstrap.cc:492] T bbae35bae8c047a6982e128ffb77ca86 P 35642aa2099c40e59bc2092604078e07: No bootstrap required, opened a new log
I20240502 07:59:43.618782 21853 ts_tablet_manager.cc:1367] T bbae35bae8c047a6982e128ffb77ca86 P 35642aa2099c40e59bc2092604078e07: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:43.619217 21853 raft_consensus.cc:348] T bbae35bae8c047a6982e128ffb77ca86 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:43.619318 21853 raft_consensus.cc:374] T bbae35bae8c047a6982e128ffb77ca86 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:43.619283 20781 tablet_service.cc:1467] Processing CreateTablet for tablet bbae35bae8c047a6982e128ffb77ca86 (DEFAULT_TABLE table=rebalance_test_table_3 [id=1388635dd26f4d3791ee92dfbf8d374a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:43.619347 21853 raft_consensus.cc:729] T bbae35bae8c047a6982e128ffb77ca86 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35642aa2099c40e59bc2092604078e07, State: Initialized, Role: FOLLOWER
I20240502 07:59:43.619496 20781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbae35bae8c047a6982e128ffb77ca86. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:43.619450 21853 consensus_queue.cc:260] T bbae35bae8c047a6982e128ffb77ca86 P 35642aa2099c40e59bc2092604078e07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:43.619793 21853 ts_tablet_manager.cc:1398] T bbae35bae8c047a6982e128ffb77ca86 P 35642aa2099c40e59bc2092604078e07: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:43.619940 21853 tablet_bootstrap.cc:492] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07: Bootstrap starting.
I20240502 07:59:43.620546 21861 tablet_bootstrap.cc:492] T e61fe3cb61c240f9b2c898aff1951144 P c15e23a34059456788f9775031e27ec0: Bootstrap starting.
I20240502 07:59:43.620707 21853 tablet_bootstrap.cc:654] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:43.621353 21861 tablet_bootstrap.cc:654] T e61fe3cb61c240f9b2c898aff1951144 P c15e23a34059456788f9775031e27ec0: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:43.621402 20782 tablet_service.cc:1467] Processing CreateTablet for tablet c3ca6f24c92d4273ac6c9d83161813d7 (DEFAULT_TABLE table=rebalance_test_table_3 [id=1388635dd26f4d3791ee92dfbf8d374a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:43.621587 20782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3ca6f24c92d4273ac6c9d83161813d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:43.622009 21855 tablet_bootstrap.cc:492] T e61fe3cb61c240f9b2c898aff1951144 P b7604c0eb3bf4a7cb260b564a94a1191: No bootstrap required, opened a new log
I20240502 07:59:43.622092 21855 ts_tablet_manager.cc:1367] T e61fe3cb61c240f9b2c898aff1951144 P b7604c0eb3bf4a7cb260b564a94a1191: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20240502 07:59:43.622171 21853 tablet_bootstrap.cc:492] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07: No bootstrap required, opened a new log
I20240502 07:59:43.622176 21852 tablet_bootstrap.cc:492] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff: No bootstrap required, opened a new log
I20240502 07:59:43.622236 21853 ts_tablet_manager.cc:1367] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:43.622243 21852 ts_tablet_manager.cc:1367] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20240502 07:59:43.622509 21855 raft_consensus.cc:348] T e61fe3cb61c240f9b2c898aff1951144 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:43.622566 21853 raft_consensus.cc:348] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:43.622618 21855 raft_consensus.cc:374] T e61fe3cb61c240f9b2c898aff1951144 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:43.622658 21853 raft_consensus.cc:374] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:43.622661 21855 raft_consensus.cc:729] T e61fe3cb61c240f9b2c898aff1951144 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7604c0eb3bf4a7cb260b564a94a1191, State: Initialized, Role: FOLLOWER
I20240502 07:59:43.622687 21853 raft_consensus.cc:729] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35642aa2099c40e59bc2092604078e07, State: Initialized, Role: FOLLOWER
I20240502 07:59:43.622637 21852 raft_consensus.cc:348] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:43.622732 21852 raft_consensus.cc:374] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:43.622759 21852 raft_consensus.cc:729] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b443e68b6212450286869124684bdfff, State: Initialized, Role: FOLLOWER
I20240502 07:59:43.622781 21853 consensus_queue.cc:260] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:43.622776 21855 consensus_queue.cc:260] T e61fe3cb61c240f9b2c898aff1951144 P b7604c0eb3bf4a7cb260b564a94a1191 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:43.622865 21852 consensus_queue.cc:260] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:43.623026 21853 ts_tablet_manager.cc:1398] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:43.623148 21852 ts_tablet_manager.cc:1398] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:43.623565 20122 tablet_service.cc:1467] Processing CreateTablet for tablet e61fe3cb61c240f9b2c898aff1951144 (DEFAULT_TABLE table=rebalance_test_table_3 [id=1388635dd26f4d3791ee92dfbf8d374a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:43.623848 20122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e61fe3cb61c240f9b2c898aff1951144. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:43.624111 21861 tablet_bootstrap.cc:492] T e61fe3cb61c240f9b2c898aff1951144 P c15e23a34059456788f9775031e27ec0: No bootstrap required, opened a new log
I20240502 07:59:43.624183 21861 ts_tablet_manager.cc:1367] T e61fe3cb61c240f9b2c898aff1951144 P c15e23a34059456788f9775031e27ec0: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:59:43.624557 21861 raft_consensus.cc:348] T e61fe3cb61c240f9b2c898aff1951144 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:43.624658 21861 raft_consensus.cc:374] T e61fe3cb61c240f9b2c898aff1951144 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:43.624688 21861 raft_consensus.cc:729] T e61fe3cb61c240f9b2c898aff1951144 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c15e23a34059456788f9775031e27ec0, State: Initialized, Role: FOLLOWER
I20240502 07:59:43.624830 21861 consensus_queue.cc:260] T e61fe3cb61c240f9b2c898aff1951144 P c15e23a34059456788f9775031e27ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:43.625171 21861 ts_tablet_manager.cc:1398] T e61fe3cb61c240f9b2c898aff1951144 P c15e23a34059456788f9775031e27ec0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:43.625360 21861 tablet_bootstrap.cc:492] T bbae35bae8c047a6982e128ffb77ca86 P c15e23a34059456788f9775031e27ec0: Bootstrap starting.
I20240502 07:59:43.625905 21864 tablet_bootstrap.cc:492] T e61fe3cb61c240f9b2c898aff1951144 P 29a7180fe4574c09a8dce7c27500aab2: Bootstrap starting.
I20240502 07:59:43.626049 20123 tablet_service.cc:1467] Processing CreateTablet for tablet bbae35bae8c047a6982e128ffb77ca86 (DEFAULT_TABLE table=rebalance_test_table_3 [id=1388635dd26f4d3791ee92dfbf8d374a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:43.626205 21861 tablet_bootstrap.cc:654] T bbae35bae8c047a6982e128ffb77ca86 P c15e23a34059456788f9775031e27ec0: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:43.626245 20123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbae35bae8c047a6982e128ffb77ca86. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:43.627120 21864 tablet_bootstrap.cc:654] T e61fe3cb61c240f9b2c898aff1951144 P 29a7180fe4574c09a8dce7c27500aab2: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:43.628099 21864 tablet_bootstrap.cc:492] T e61fe3cb61c240f9b2c898aff1951144 P 29a7180fe4574c09a8dce7c27500aab2: No bootstrap required, opened a new log
I20240502 07:59:43.628171 21864 ts_tablet_manager.cc:1367] T e61fe3cb61c240f9b2c898aff1951144 P 29a7180fe4574c09a8dce7c27500aab2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240502 07:59:43.628149 20121 tablet_service.cc:1467] Processing CreateTablet for tablet c3ca6f24c92d4273ac6c9d83161813d7 (DEFAULT_TABLE table=rebalance_test_table_3 [id=1388635dd26f4d3791ee92dfbf8d374a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:43.628338 20121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3ca6f24c92d4273ac6c9d83161813d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:43.628566 21864 raft_consensus.cc:348] T e61fe3cb61c240f9b2c898aff1951144 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:43.628672 21864 raft_consensus.cc:374] T e61fe3cb61c240f9b2c898aff1951144 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:43.628698 21864 raft_consensus.cc:729] T e61fe3cb61c240f9b2c898aff1951144 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29a7180fe4574c09a8dce7c27500aab2, State: Initialized, Role: FOLLOWER
I20240502 07:59:43.623028 21855 ts_tablet_manager.cc:1398] T e61fe3cb61c240f9b2c898aff1951144 P b7604c0eb3bf4a7cb260b564a94a1191: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:43.629040 21855 tablet_bootstrap.cc:492] T bbae35bae8c047a6982e128ffb77ca86 P b7604c0eb3bf4a7cb260b564a94a1191: Bootstrap starting.
I20240502 07:59:43.628813 21864 consensus_queue.cc:260] T e61fe3cb61c240f9b2c898aff1951144 P 29a7180fe4574c09a8dce7c27500aab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:43.629596 21864 ts_tablet_manager.cc:1398] T e61fe3cb61c240f9b2c898aff1951144 P 29a7180fe4574c09a8dce7c27500aab2: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:43.629774 21864 tablet_bootstrap.cc:492] T bbae35bae8c047a6982e128ffb77ca86 P 29a7180fe4574c09a8dce7c27500aab2: Bootstrap starting.
I20240502 07:59:43.629777 21855 tablet_bootstrap.cc:654] T bbae35bae8c047a6982e128ffb77ca86 P b7604c0eb3bf4a7cb260b564a94a1191: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:43.630487 20914 tablet_service.cc:1467] Processing CreateTablet for tablet e61fe3cb61c240f9b2c898aff1951144 (DEFAULT_TABLE table=rebalance_test_table_3 [id=1388635dd26f4d3791ee92dfbf8d374a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:43.630671 21864 tablet_bootstrap.cc:654] T bbae35bae8c047a6982e128ffb77ca86 P 29a7180fe4574c09a8dce7c27500aab2: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:43.630584 20913 tablet_service.cc:1467] Processing CreateTablet for tablet c3ca6f24c92d4273ac6c9d83161813d7 (DEFAULT_TABLE table=rebalance_test_table_3 [id=1388635dd26f4d3791ee92dfbf8d374a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:43.630733 20914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e61fe3cb61c240f9b2c898aff1951144. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:43.630923 20913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3ca6f24c92d4273ac6c9d83161813d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:43.631162 20915 tablet_service.cc:1467] Processing CreateTablet for tablet bbae35bae8c047a6982e128ffb77ca86 (DEFAULT_TABLE table=rebalance_test_table_3 [id=1388635dd26f4d3791ee92dfbf8d374a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:43.631361 20915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbae35bae8c047a6982e128ffb77ca86. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:43.631654 21861 tablet_bootstrap.cc:492] T bbae35bae8c047a6982e128ffb77ca86 P c15e23a34059456788f9775031e27ec0: No bootstrap required, opened a new log
I20240502 07:59:43.631721 21861 ts_tablet_manager.cc:1367] T bbae35bae8c047a6982e128ffb77ca86 P c15e23a34059456788f9775031e27ec0: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240502 07:59:43.632066 21861 raft_consensus.cc:348] T bbae35bae8c047a6982e128ffb77ca86 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:43.632169 21861 raft_consensus.cc:374] T bbae35bae8c047a6982e128ffb77ca86 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:43.632196 21861 raft_consensus.cc:729] T bbae35bae8c047a6982e128ffb77ca86 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c15e23a34059456788f9775031e27ec0, State: Initialized, Role: FOLLOWER
I20240502 07:59:43.632342 21861 consensus_queue.cc:260] T bbae35bae8c047a6982e128ffb77ca86 P c15e23a34059456788f9775031e27ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:43.632690 21861 ts_tablet_manager.cc:1398] T bbae35bae8c047a6982e128ffb77ca86 P c15e23a34059456788f9775031e27ec0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:43.632822 21861 tablet_bootstrap.cc:492] T c3ca6f24c92d4273ac6c9d83161813d7 P c15e23a34059456788f9775031e27ec0: Bootstrap starting.
I20240502 07:59:43.633721 21861 tablet_bootstrap.cc:654] T c3ca6f24c92d4273ac6c9d83161813d7 P c15e23a34059456788f9775031e27ec0: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:43.634322 21868 tablet_bootstrap.cc:492] T c3ca6f24c92d4273ac6c9d83161813d7 P b5f471f8b5944986a7d878cf6ac550b8: Bootstrap starting.
I20240502 07:59:43.634632 21864 tablet_bootstrap.cc:492] T bbae35bae8c047a6982e128ffb77ca86 P 29a7180fe4574c09a8dce7c27500aab2: No bootstrap required, opened a new log
I20240502 07:59:43.634706 21864 ts_tablet_manager.cc:1367] T bbae35bae8c047a6982e128ffb77ca86 P 29a7180fe4574c09a8dce7c27500aab2: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 07:59:43.634969 20519 tablet_service.cc:1467] Processing CreateTablet for tablet c3ca6f24c92d4273ac6c9d83161813d7 (DEFAULT_TABLE table=rebalance_test_table_3 [id=1388635dd26f4d3791ee92dfbf8d374a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:43.635128 21864 raft_consensus.cc:348] T bbae35bae8c047a6982e128ffb77ca86 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:43.635227 21864 raft_consensus.cc:374] T bbae35bae8c047a6982e128ffb77ca86 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:43.635262 21864 raft_consensus.cc:729] T bbae35bae8c047a6982e128ffb77ca86 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29a7180fe4574c09a8dce7c27500aab2, State: Initialized, Role: FOLLOWER
I20240502 07:59:43.635277 20519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3ca6f24c92d4273ac6c9d83161813d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:43.635367 21864 consensus_queue.cc:260] T bbae35bae8c047a6982e128ffb77ca86 P 29a7180fe4574c09a8dce7c27500aab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:43.635634 21864 ts_tablet_manager.cc:1398] T bbae35bae8c047a6982e128ffb77ca86 P 29a7180fe4574c09a8dce7c27500aab2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:43.635780 21864 tablet_bootstrap.cc:492] T c3ca6f24c92d4273ac6c9d83161813d7 P 29a7180fe4574c09a8dce7c27500aab2: Bootstrap starting.
I20240502 07:59:43.636580 21864 tablet_bootstrap.cc:654] T c3ca6f24c92d4273ac6c9d83161813d7 P 29a7180fe4574c09a8dce7c27500aab2: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:43.636695 21855 tablet_bootstrap.cc:492] T bbae35bae8c047a6982e128ffb77ca86 P b7604c0eb3bf4a7cb260b564a94a1191: No bootstrap required, opened a new log
I20240502 07:59:43.636773 21855 ts_tablet_manager.cc:1367] T bbae35bae8c047a6982e128ffb77ca86 P b7604c0eb3bf4a7cb260b564a94a1191: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.002s
I20240502 07:59:43.637044 21851 tablet_bootstrap.cc:492] T bbae35bae8c047a6982e128ffb77ca86 P e951f009258e473cb57cf8773c7975d4: Bootstrap starting.
I20240502 07:59:43.637156 21855 raft_consensus.cc:348] T bbae35bae8c047a6982e128ffb77ca86 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:43.637274 21855 raft_consensus.cc:374] T bbae35bae8c047a6982e128ffb77ca86 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:43.637312 21855 raft_consensus.cc:729] T bbae35bae8c047a6982e128ffb77ca86 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7604c0eb3bf4a7cb260b564a94a1191, State: Initialized, Role: FOLLOWER
I20240502 07:59:43.637423 21855 consensus_queue.cc:260] T bbae35bae8c047a6982e128ffb77ca86 P b7604c0eb3bf4a7cb260b564a94a1191 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:43.637708 21855 ts_tablet_manager.cc:1398] T bbae35bae8c047a6982e128ffb77ca86 P b7604c0eb3bf4a7cb260b564a94a1191: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:43.637884 21855 tablet_bootstrap.cc:492] T c3ca6f24c92d4273ac6c9d83161813d7 P b7604c0eb3bf4a7cb260b564a94a1191: Bootstrap starting.
I20240502 07:59:43.638293 21868 tablet_bootstrap.cc:654] T c3ca6f24c92d4273ac6c9d83161813d7 P b5f471f8b5944986a7d878cf6ac550b8: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:43.638304 21423 raft_consensus.cc:482] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:43.638381 21423 raft_consensus.cc:504] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:43.638445 21851 tablet_bootstrap.cc:654] T bbae35bae8c047a6982e128ffb77ca86 P e951f009258e473cb57cf8773c7975d4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:43.638634 21855 tablet_bootstrap.cc:654] T c3ca6f24c92d4273ac6c9d83161813d7 P b7604c0eb3bf4a7cb260b564a94a1191: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:43.639024 21423 leader_election.cc:288] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703), b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331), 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587), c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085), e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315), b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577)
I20240502 07:59:43.639233 20273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3ca6f24c92d4273ac6c9d83161813d7" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" is_pre_election: true
I20240502 07:59:43.639791 21851 tablet_bootstrap.cc:492] T bbae35bae8c047a6982e128ffb77ca86 P e951f009258e473cb57cf8773c7975d4: No bootstrap required, opened a new log
W20240502 07:59:43.639793 20606 leader_election.cc:341] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331): Illegal state: must be running to vote when last-logged opid is not known
I20240502 07:59:43.639883 21851 ts_tablet_manager.cc:1367] T bbae35bae8c047a6982e128ffb77ca86 P e951f009258e473cb57cf8773c7975d4: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:59:43.639984 21861 tablet_bootstrap.cc:492] T c3ca6f24c92d4273ac6c9d83161813d7 P c15e23a34059456788f9775031e27ec0: No bootstrap required, opened a new log
I20240502 07:59:43.640067 21861 ts_tablet_manager.cc:1367] T c3ca6f24c92d4273ac6c9d83161813d7 P c15e23a34059456788f9775031e27ec0: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 07:59:43.640300 21851 raft_consensus.cc:348] T bbae35bae8c047a6982e128ffb77ca86 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:43.640429 21851 raft_consensus.cc:374] T bbae35bae8c047a6982e128ffb77ca86 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:43.640400 21861 raft_consensus.cc:348] T c3ca6f24c92d4273ac6c9d83161813d7 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:43.640462 21851 raft_consensus.cc:729] T bbae35bae8c047a6982e128ffb77ca86 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e951f009258e473cb57cf8773c7975d4, State: Initialized, Role: FOLLOWER
I20240502 07:59:43.640499 21861 raft_consensus.cc:374] T c3ca6f24c92d4273ac6c9d83161813d7 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:43.640532 21861 raft_consensus.cc:729] T c3ca6f24c92d4273ac6c9d83161813d7 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c15e23a34059456788f9775031e27ec0, State: Initialized, Role: FOLLOWER
I20240502 07:59:43.640609 21851 consensus_queue.cc:260] T bbae35bae8c047a6982e128ffb77ca86 P e951f009258e473cb57cf8773c7975d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:43.640684 20802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3ca6f24c92d4273ac6c9d83161813d7" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15e23a34059456788f9775031e27ec0" is_pre_election: true
I20240502 07:59:43.640651 21861 consensus_queue.cc:260] T c3ca6f24c92d4273ac6c9d83161813d7 P c15e23a34059456788f9775031e27ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:43.640959 20802 raft_consensus.cc:2454] T c3ca6f24c92d4273ac6c9d83161813d7 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b443e68b6212450286869124684bdfff in term 0.
I20240502 07:59:43.641261 21864 tablet_bootstrap.cc:492] T c3ca6f24c92d4273ac6c9d83161813d7 P 29a7180fe4574c09a8dce7c27500aab2: No bootstrap required, opened a new log
I20240502 07:59:43.641284 21851 ts_tablet_manager.cc:1398] T bbae35bae8c047a6982e128ffb77ca86 P e951f009258e473cb57cf8773c7975d4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:43.641346 21864 ts_tablet_manager.cc:1367] T c3ca6f24c92d4273ac6c9d83161813d7 P 29a7180fe4574c09a8dce7c27500aab2: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240502 07:59:43.641433 21851 tablet_bootstrap.cc:492] T e61fe3cb61c240f9b2c898aff1951144 P e951f009258e473cb57cf8773c7975d4: Bootstrap starting.
I20240502 07:59:43.641505 21861 ts_tablet_manager.cc:1398] T c3ca6f24c92d4273ac6c9d83161813d7 P c15e23a34059456788f9775031e27ec0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:43.641577 20539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3ca6f24c92d4273ac6c9d83161813d7" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e951f009258e473cb57cf8773c7975d4" is_pre_election: true
I20240502 07:59:43.641729 21864 raft_consensus.cc:348] T c3ca6f24c92d4273ac6c9d83161813d7 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:43.641821 21864 raft_consensus.cc:374] T c3ca6f24c92d4273ac6c9d83161813d7 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:43.641849 21864 raft_consensus.cc:729] T c3ca6f24c92d4273ac6c9d83161813d7 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29a7180fe4574c09a8dce7c27500aab2, State: Initialized, Role: FOLLOWER
I20240502 07:59:43.641896 20407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3ca6f24c92d4273ac6c9d83161813d7" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35642aa2099c40e59bc2092604078e07" is_pre_election: true
I20240502 07:59:43.641947 21864 consensus_queue.cc:260] T c3ca6f24c92d4273ac6c9d83161813d7 P 29a7180fe4574c09a8dce7c27500aab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:43.642019 20407 raft_consensus.cc:2454] T c3ca6f24c92d4273ac6c9d83161813d7 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b443e68b6212450286869124684bdfff in term 0.
I20240502 07:59:43.642175 21851 tablet_bootstrap.cc:654] T e61fe3cb61c240f9b2c898aff1951144 P e951f009258e473cb57cf8773c7975d4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:43.642218 21864 ts_tablet_manager.cc:1398] T c3ca6f24c92d4273ac6c9d83161813d7 P 29a7180fe4574c09a8dce7c27500aab2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:43.642657 20935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3ca6f24c92d4273ac6c9d83161813d7" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f471f8b5944986a7d878cf6ac550b8" is_pre_election: true
I20240502 07:59:43.642874 20142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3ca6f24c92d4273ac6c9d83161813d7" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7180fe4574c09a8dce7c27500aab2" is_pre_election: true
I20240502 07:59:43.642976 20142 raft_consensus.cc:2454] T c3ca6f24c92d4273ac6c9d83161813d7 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b443e68b6212450286869124684bdfff in term 0.
W20240502 07:59:43.643230 20607 leader_election.cc:341] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577): Illegal state: must be running to vote when last-logged opid is not known
W20240502 07:59:43.643324 20605 leader_election.cc:341] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315): Illegal state: must be running to vote when last-logged opid is not known
I20240502 07:59:43.643501 20605 leader_election.cc:302] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 7 responses out of 7 voters: 4 yes votes; 3 no votes. yes voters: 29a7180fe4574c09a8dce7c27500aab2, 35642aa2099c40e59bc2092604078e07, b443e68b6212450286869124684bdfff, c15e23a34059456788f9775031e27ec0; no voters: b5f471f8b5944986a7d878cf6ac550b8, b7604c0eb3bf4a7cb260b564a94a1191, e951f009258e473cb57cf8773c7975d4
I20240502 07:59:43.644165 21855 tablet_bootstrap.cc:492] T c3ca6f24c92d4273ac6c9d83161813d7 P b7604c0eb3bf4a7cb260b564a94a1191: No bootstrap required, opened a new log
I20240502 07:59:43.644250 21855 ts_tablet_manager.cc:1367] T c3ca6f24c92d4273ac6c9d83161813d7 P b7604c0eb3bf4a7cb260b564a94a1191: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20240502 07:59:43.644618 21855 raft_consensus.cc:348] T c3ca6f24c92d4273ac6c9d83161813d7 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:43.644721 21855 raft_consensus.cc:374] T c3ca6f24c92d4273ac6c9d83161813d7 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:43.644764 21855 raft_consensus.cc:729] T c3ca6f24c92d4273ac6c9d83161813d7 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7604c0eb3bf4a7cb260b564a94a1191, State: Initialized, Role: FOLLOWER
I20240502 07:59:43.644886 21855 consensus_queue.cc:260] T c3ca6f24c92d4273ac6c9d83161813d7 P b7604c0eb3bf4a7cb260b564a94a1191 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:43.645054 21868 tablet_bootstrap.cc:492] T c3ca6f24c92d4273ac6c9d83161813d7 P b5f471f8b5944986a7d878cf6ac550b8: No bootstrap required, opened a new log
I20240502 07:59:43.645139 21868 ts_tablet_manager.cc:1367] T c3ca6f24c92d4273ac6c9d83161813d7 P b5f471f8b5944986a7d878cf6ac550b8: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20240502 07:59:43.645165 21855 ts_tablet_manager.cc:1398] T c3ca6f24c92d4273ac6c9d83161813d7 P b7604c0eb3bf4a7cb260b564a94a1191: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:43.645493 21868 raft_consensus.cc:348] T c3ca6f24c92d4273ac6c9d83161813d7 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:43.645617 21868 raft_consensus.cc:374] T c3ca6f24c92d4273ac6c9d83161813d7 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:43.645646 21868 raft_consensus.cc:729] T c3ca6f24c92d4273ac6c9d83161813d7 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5f471f8b5944986a7d878cf6ac550b8, State: Initialized, Role: FOLLOWER
I20240502 07:59:43.645761 21868 consensus_queue.cc:260] T c3ca6f24c92d4273ac6c9d83161813d7 P b5f471f8b5944986a7d878cf6ac550b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:43.646013 21868 ts_tablet_manager.cc:1398] T c3ca6f24c92d4273ac6c9d83161813d7 P b5f471f8b5944986a7d878cf6ac550b8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:43.646425 21868 tablet_bootstrap.cc:492] T bbae35bae8c047a6982e128ffb77ca86 P b5f471f8b5944986a7d878cf6ac550b8: Bootstrap starting.
I20240502 07:59:43.646857 21423 raft_consensus.cc:2781] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:43.646919 21423 raft_consensus.cc:482] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:43.646956 21423 raft_consensus.cc:3037] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:43.647225 21868 tablet_bootstrap.cc:654] T bbae35bae8c047a6982e128ffb77ca86 P b5f471f8b5944986a7d878cf6ac550b8: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:43.647769 21423 raft_consensus.cc:504] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:43.648192 21868 tablet_bootstrap.cc:492] T bbae35bae8c047a6982e128ffb77ca86 P b5f471f8b5944986a7d878cf6ac550b8: No bootstrap required, opened a new log
I20240502 07:59:43.648280 21868 ts_tablet_manager.cc:1367] T bbae35bae8c047a6982e128ffb77ca86 P b5f471f8b5944986a7d878cf6ac550b8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:43.648336 21423 leader_election.cc:288] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [CANDIDATE]: Term 1 election: Requested vote from peers 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703), b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331), 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587), c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085), e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315), b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577)
I20240502 07:59:43.648702 20273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3ca6f24c92d4273ac6c9d83161813d7" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7604c0eb3bf4a7cb260b564a94a1191"
I20240502 07:59:43.648697 21868 raft_consensus.cc:348] T bbae35bae8c047a6982e128ffb77ca86 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:43.648794 20273 raft_consensus.cc:3037] T c3ca6f24c92d4273ac6c9d83161813d7 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:43.648800 21851 tablet_bootstrap.cc:492] T e61fe3cb61c240f9b2c898aff1951144 P e951f009258e473cb57cf8773c7975d4: No bootstrap required, opened a new log
I20240502 07:59:43.648804 21868 raft_consensus.cc:374] T bbae35bae8c047a6982e128ffb77ca86 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:43.648837 21868 raft_consensus.cc:729] T bbae35bae8c047a6982e128ffb77ca86 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5f471f8b5944986a7d878cf6ac550b8, State: Initialized, Role: FOLLOWER
I20240502 07:59:43.648870 21851 ts_tablet_manager.cc:1367] T e61fe3cb61c240f9b2c898aff1951144 P e951f009258e473cb57cf8773c7975d4: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240502 07:59:43.648962 21868 consensus_queue.cc:260] T bbae35bae8c047a6982e128ffb77ca86 P b5f471f8b5944986a7d878cf6ac550b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:43.649006 20539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3ca6f24c92d4273ac6c9d83161813d7" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e951f009258e473cb57cf8773c7975d4"
I20240502 07:59:43.649235 21851 raft_consensus.cc:348] T e61fe3cb61c240f9b2c898aff1951144 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:43.649299 21868 ts_tablet_manager.cc:1398] T bbae35bae8c047a6982e128ffb77ca86 P b5f471f8b5944986a7d878cf6ac550b8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:43.649340 21851 raft_consensus.cc:374] T e61fe3cb61c240f9b2c898aff1951144 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:43.649369 21851 raft_consensus.cc:729] T e61fe3cb61c240f9b2c898aff1951144 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e951f009258e473cb57cf8773c7975d4, State: Initialized, Role: FOLLOWER
I20240502 07:59:43.649413 20142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3ca6f24c92d4273ac6c9d83161813d7" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7180fe4574c09a8dce7c27500aab2"
I20240502 07:59:43.649499 20142 raft_consensus.cc:3037] T c3ca6f24c92d4273ac6c9d83161813d7 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:43.649482 21851 consensus_queue.cc:260] T e61fe3cb61c240f9b2c898aff1951144 P e951f009258e473cb57cf8773c7975d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:43.649663 20273 raft_consensus.cc:2454] T c3ca6f24c92d4273ac6c9d83161813d7 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b443e68b6212450286869124684bdfff in term 1.
I20240502 07:59:43.649747 21851 ts_tablet_manager.cc:1398] T e61fe3cb61c240f9b2c898aff1951144 P e951f009258e473cb57cf8773c7975d4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:43.649893 21851 tablet_bootstrap.cc:492] T c3ca6f24c92d4273ac6c9d83161813d7 P e951f009258e473cb57cf8773c7975d4: Bootstrap starting.
I20240502 07:59:43.649935 20802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3ca6f24c92d4273ac6c9d83161813d7" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15e23a34059456788f9775031e27ec0"
I20240502 07:59:43.650024 20802 raft_consensus.cc:3037] T c3ca6f24c92d4273ac6c9d83161813d7 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:43.650300 20142 raft_consensus.cc:2454] T c3ca6f24c92d4273ac6c9d83161813d7 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b443e68b6212450286869124684bdfff in term 1.
I20240502 07:59:43.650664 21851 tablet_bootstrap.cc:654] T c3ca6f24c92d4273ac6c9d83161813d7 P e951f009258e473cb57cf8773c7975d4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:43.650776 21509 raft_consensus.cc:482] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:43.650873 20802 raft_consensus.cc:2454] T c3ca6f24c92d4273ac6c9d83161813d7 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b443e68b6212450286869124684bdfff in term 1.
I20240502 07:59:43.650875 21509 raft_consensus.cc:504] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
W20240502 07:59:43.650946 20605 leader_election.cc:341] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315): Illegal state: must be running to vote when last-logged opid is not known
I20240502 07:59:43.650755 20407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3ca6f24c92d4273ac6c9d83161813d7" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35642aa2099c40e59bc2092604078e07"
I20240502 07:59:43.651108 20407 raft_consensus.cc:3037] T c3ca6f24c92d4273ac6c9d83161813d7 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:43.651235 20935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3ca6f24c92d4273ac6c9d83161813d7" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f471f8b5944986a7d878cf6ac550b8"
I20240502 07:59:43.651348 20935 raft_consensus.cc:3037] T c3ca6f24c92d4273ac6c9d83161813d7 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:43.651495 21509 leader_election.cc:288] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b443e68b6212450286869124684bdfff (127.3.139.69:37923), c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085), e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315), b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577), b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331), 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703)
I20240502 07:59:43.651974 20407 raft_consensus.cc:2454] T c3ca6f24c92d4273ac6c9d83161813d7 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b443e68b6212450286869124684bdfff in term 1.
I20240502 07:59:43.652014 21868 tablet_bootstrap.cc:492] T e61fe3cb61c240f9b2c898aff1951144 P b5f471f8b5944986a7d878cf6ac550b8: Bootstrap starting.
I20240502 07:59:43.652156 20935 raft_consensus.cc:2454] T c3ca6f24c92d4273ac6c9d83161813d7 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b443e68b6212450286869124684bdfff in term 1.
I20240502 07:59:43.652415 20669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e61fe3cb61c240f9b2c898aff1951144" candidate_uuid: "35642aa2099c40e59bc2092604078e07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b443e68b6212450286869124684bdfff" is_pre_election: true
I20240502 07:59:43.652534 20669 raft_consensus.cc:2454] T e61fe3cb61c240f9b2c898aff1951144 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35642aa2099c40e59bc2092604078e07 in term 0.
I20240502 07:59:43.652820 21868 tablet_bootstrap.cc:654] T e61fe3cb61c240f9b2c898aff1951144 P b5f471f8b5944986a7d878cf6ac550b8: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:43.652947 20273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e61fe3cb61c240f9b2c898aff1951144" candidate_uuid: "35642aa2099c40e59bc2092604078e07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" is_pre_election: true
I20240502 07:59:43.653051 20273 raft_consensus.cc:2454] T e61fe3cb61c240f9b2c898aff1951144 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35642aa2099c40e59bc2092604078e07 in term 0.
I20240502 07:59:43.653666 20539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e61fe3cb61c240f9b2c898aff1951144" candidate_uuid: "35642aa2099c40e59bc2092604078e07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e951f009258e473cb57cf8773c7975d4" is_pre_election: true
I20240502 07:59:43.653785 20539 raft_consensus.cc:2454] T e61fe3cb61c240f9b2c898aff1951144 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35642aa2099c40e59bc2092604078e07 in term 0.
I20240502 07:59:43.654269 20802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e61fe3cb61c240f9b2c898aff1951144" candidate_uuid: "35642aa2099c40e59bc2092604078e07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15e23a34059456788f9775031e27ec0" is_pre_election: true
I20240502 07:59:43.654340 21868 tablet_bootstrap.cc:492] T e61fe3cb61c240f9b2c898aff1951144 P b5f471f8b5944986a7d878cf6ac550b8: No bootstrap required, opened a new log
I20240502 07:59:43.654383 20802 raft_consensus.cc:2454] T e61fe3cb61c240f9b2c898aff1951144 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35642aa2099c40e59bc2092604078e07 in term 0.
I20240502 07:59:43.654362 20342 leader_election.cc:302] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 35642aa2099c40e59bc2092604078e07, b443e68b6212450286869124684bdfff, b7604c0eb3bf4a7cb260b564a94a1191, e951f009258e473cb57cf8773c7975d4; no voters:
I20240502 07:59:43.654412 21868 ts_tablet_manager.cc:1367] T e61fe3cb61c240f9b2c898aff1951144 P b5f471f8b5944986a7d878cf6ac550b8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:43.654790 21868 raft_consensus.cc:348] T e61fe3cb61c240f9b2c898aff1951144 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:43.654901 21868 raft_consensus.cc:374] T e61fe3cb61c240f9b2c898aff1951144 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:43.654937 21868 raft_consensus.cc:729] T e61fe3cb61c240f9b2c898aff1951144 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5f471f8b5944986a7d878cf6ac550b8, State: Initialized, Role: FOLLOWER
I20240502 07:59:43.655066 21868 consensus_queue.cc:260] T e61fe3cb61c240f9b2c898aff1951144 P b5f471f8b5944986a7d878cf6ac550b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:43.655375 21868 ts_tablet_manager.cc:1398] T e61fe3cb61c240f9b2c898aff1951144 P b5f471f8b5944986a7d878cf6ac550b8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:43.652426 20607 leader_election.cc:302] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 29a7180fe4574c09a8dce7c27500aab2, b443e68b6212450286869124684bdfff, b5f471f8b5944986a7d878cf6ac550b8, b7604c0eb3bf4a7cb260b564a94a1191; no voters: e951f009258e473cb57cf8773c7975d4
I20240502 07:59:43.656092 21879 raft_consensus.cc:2781] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:43.656177 21879 raft_consensus.cc:686] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [term 1 LEADER]: Becoming Leader. State: Replica: b443e68b6212450286869124684bdfff, State: Running, Role: LEADER
I20240502 07:59:43.656323 21879 consensus_queue.cc:237] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:43.656838 21509 raft_consensus.cc:2781] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:43.656896 21509 raft_consensus.cc:482] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:43.656936 21509 raft_consensus.cc:3037] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:43.657727 21509 raft_consensus.cc:504] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:43.658282 21509 leader_election.cc:288] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [CANDIDATE]: Term 1 election: Requested vote from peers b443e68b6212450286869124684bdfff (127.3.139.69:37923), c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085), e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315), b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577), b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331), 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703)
I20240502 07:59:43.658438 20142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e61fe3cb61c240f9b2c898aff1951144" candidate_uuid: "35642aa2099c40e59bc2092604078e07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7180fe4574c09a8dce7c27500aab2" is_pre_election: true
I20240502 07:59:43.658460 20669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e61fe3cb61c240f9b2c898aff1951144" candidate_uuid: "35642aa2099c40e59bc2092604078e07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b443e68b6212450286869124684bdfff"
I20240502 07:59:43.658552 20669 raft_consensus.cc:3037] T e61fe3cb61c240f9b2c898aff1951144 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:43.658548 20142 raft_consensus.cc:2454] T e61fe3cb61c240f9b2c898aff1951144 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35642aa2099c40e59bc2092604078e07 in term 0.
I20240502 07:59:43.658565 20802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e61fe3cb61c240f9b2c898aff1951144" candidate_uuid: "35642aa2099c40e59bc2092604078e07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15e23a34059456788f9775031e27ec0"
I20240502 07:59:43.658650 20802 raft_consensus.cc:3037] T e61fe3cb61c240f9b2c898aff1951144 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:43.658707 20143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e61fe3cb61c240f9b2c898aff1951144" candidate_uuid: "35642aa2099c40e59bc2092604078e07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7180fe4574c09a8dce7c27500aab2"
I20240502 07:59:43.658717 20935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e61fe3cb61c240f9b2c898aff1951144" candidate_uuid: "35642aa2099c40e59bc2092604078e07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f471f8b5944986a7d878cf6ac550b8" is_pre_election: true
I20240502 07:59:43.658798 20143 raft_consensus.cc:3037] T e61fe3cb61c240f9b2c898aff1951144 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:43.658809 20935 raft_consensus.cc:2454] T e61fe3cb61c240f9b2c898aff1951144 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35642aa2099c40e59bc2092604078e07 in term 0.
I20240502 07:59:43.658892 20539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e61fe3cb61c240f9b2c898aff1951144" candidate_uuid: "35642aa2099c40e59bc2092604078e07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e951f009258e473cb57cf8773c7975d4"
I20240502 07:59:43.658973 20539 raft_consensus.cc:3037] T e61fe3cb61c240f9b2c898aff1951144 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:43.659484 20802 raft_consensus.cc:2454] T e61fe3cb61c240f9b2c898aff1951144 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35642aa2099c40e59bc2092604078e07 in term 1.
I20240502 07:59:43.659621 20143 raft_consensus.cc:2454] T e61fe3cb61c240f9b2c898aff1951144 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35642aa2099c40e59bc2092604078e07 in term 1.
I20240502 07:59:43.659855 20539 raft_consensus.cc:2454] T e61fe3cb61c240f9b2c898aff1951144 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35642aa2099c40e59bc2092604078e07 in term 1.
I20240502 07:59:43.660089 20341 leader_election.cc:302] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 29a7180fe4574c09a8dce7c27500aab2, 35642aa2099c40e59bc2092604078e07, c15e23a34059456788f9775031e27ec0, e951f009258e473cb57cf8773c7975d4; no voters:
I20240502 07:59:43.660228 20934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e61fe3cb61c240f9b2c898aff1951144" candidate_uuid: "35642aa2099c40e59bc2092604078e07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f471f8b5944986a7d878cf6ac550b8"
I20240502 07:59:43.660256 21509 raft_consensus.cc:2781] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:43.660277 20273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e61fe3cb61c240f9b2c898aff1951144" candidate_uuid: "35642aa2099c40e59bc2092604078e07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7604c0eb3bf4a7cb260b564a94a1191"
I20240502 07:59:43.660318 20934 raft_consensus.cc:3037] T e61fe3cb61c240f9b2c898aff1951144 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:43.660370 20273 raft_consensus.cc:3037] T e61fe3cb61c240f9b2c898aff1951144 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:43.660423 21509 raft_consensus.cc:686] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [term 1 LEADER]: Becoming Leader. State: Replica: 35642aa2099c40e59bc2092604078e07, State: Running, Role: LEADER
I20240502 07:59:43.660559 21509 consensus_queue.cc:237] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:43.661098 20934 raft_consensus.cc:2454] T e61fe3cb61c240f9b2c898aff1951144 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35642aa2099c40e59bc2092604078e07 in term 1.
I20240502 07:59:43.661193 20273 raft_consensus.cc:2454] T e61fe3cb61c240f9b2c898aff1951144 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35642aa2099c40e59bc2092604078e07 in term 1.
I20240502 07:59:43.661773 20000 catalog_manager.cc:5484] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff reported cstate change: term changed from 0 to 1, leader changed from <none> to b443e68b6212450286869124684bdfff (127.3.139.69). New cstate: current_term: 1 leader_uuid: "b443e68b6212450286869124684bdfff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:43.662516 20001 catalog_manager.cc:5484] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 reported cstate change: term changed from 0 to 1, leader changed from <none> to 35642aa2099c40e59bc2092604078e07 (127.3.139.67). New cstate: current_term: 1 leader_uuid: "35642aa2099c40e59bc2092604078e07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: HEALTHY } } }
I20240502 07:59:43.675350 21851 tablet_bootstrap.cc:492] T c3ca6f24c92d4273ac6c9d83161813d7 P e951f009258e473cb57cf8773c7975d4: No bootstrap required, opened a new log
I20240502 07:59:43.675460 21851 ts_tablet_manager.cc:1367] T c3ca6f24c92d4273ac6c9d83161813d7 P e951f009258e473cb57cf8773c7975d4: Time spent bootstrapping tablet: real 0.026s user 0.002s sys 0.000s
I20240502 07:59:43.675930 20669 raft_consensus.cc:2454] T e61fe3cb61c240f9b2c898aff1951144 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35642aa2099c40e59bc2092604078e07 in term 1.
I20240502 07:59:43.675900 21851 raft_consensus.cc:348] T c3ca6f24c92d4273ac6c9d83161813d7 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:43.676015 21851 raft_consensus.cc:374] T c3ca6f24c92d4273ac6c9d83161813d7 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:43.676051 21851 raft_consensus.cc:729] T c3ca6f24c92d4273ac6c9d83161813d7 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e951f009258e473cb57cf8773c7975d4, State: Initialized, Role: FOLLOWER
I20240502 07:59:43.676167 21851 consensus_queue.cc:260] T c3ca6f24c92d4273ac6c9d83161813d7 P e951f009258e473cb57cf8773c7975d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:43.676470 21851 ts_tablet_manager.cc:1398] T c3ca6f24c92d4273ac6c9d83161813d7 P e951f009258e473cb57cf8773c7975d4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:43.792798 21423 raft_consensus.cc:482] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:43.792917 21423 raft_consensus.cc:504] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:43.793563 21423 leader_election.cc:288] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085), e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315), b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577), 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703), b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331), 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587)
I20240502 07:59:43.793825 20802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbae35bae8c047a6982e128ffb77ca86" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15e23a34059456788f9775031e27ec0" is_pre_election: true
I20240502 07:59:43.793869 20539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbae35bae8c047a6982e128ffb77ca86" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e951f009258e473cb57cf8773c7975d4" is_pre_election: true
I20240502 07:59:43.793970 20802 raft_consensus.cc:2454] T bbae35bae8c047a6982e128ffb77ca86 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b443e68b6212450286869124684bdfff in term 0.
I20240502 07:59:43.793973 20539 raft_consensus.cc:2454] T bbae35bae8c047a6982e128ffb77ca86 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b443e68b6212450286869124684bdfff in term 0.
I20240502 07:59:43.793992 20273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbae35bae8c047a6982e128ffb77ca86" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" is_pre_election: true
I20240502 07:59:43.794124 20273 raft_consensus.cc:2454] T bbae35bae8c047a6982e128ffb77ca86 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b443e68b6212450286869124684bdfff in term 0.
I20240502 07:59:43.794247 20934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbae35bae8c047a6982e128ffb77ca86" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f471f8b5944986a7d878cf6ac550b8" is_pre_election: true
I20240502 07:59:43.794377 20934 raft_consensus.cc:2454] T bbae35bae8c047a6982e128ffb77ca86 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b443e68b6212450286869124684bdfff in term 0.
I20240502 07:59:43.794387 20407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbae35bae8c047a6982e128ffb77ca86" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35642aa2099c40e59bc2092604078e07" is_pre_election: true
I20240502 07:59:43.794395 20143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbae35bae8c047a6982e128ffb77ca86" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7180fe4574c09a8dce7c27500aab2" is_pre_election: true
I20240502 07:59:43.794502 20407 raft_consensus.cc:2454] T bbae35bae8c047a6982e128ffb77ca86 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b443e68b6212450286869124684bdfff in term 0.
I20240502 07:59:43.794523 20143 raft_consensus.cc:2454] T bbae35bae8c047a6982e128ffb77ca86 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b443e68b6212450286869124684bdfff in term 0.
I20240502 07:59:43.794700 20605 leader_election.cc:302] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: b443e68b6212450286869124684bdfff, b7604c0eb3bf4a7cb260b564a94a1191, c15e23a34059456788f9775031e27ec0, e951f009258e473cb57cf8773c7975d4; no voters:
I20240502 07:59:43.794849 21423 raft_consensus.cc:2781] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:43.794898 21423 raft_consensus.cc:482] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:43.794935 21423 raft_consensus.cc:3037] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:43.795871 21423 raft_consensus.cc:504] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:43.796339 21423 leader_election.cc:288] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [CANDIDATE]: Term 1 election: Requested vote from peers c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085), e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315), b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577), 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703), b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331), 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587)
I20240502 07:59:43.796521 20802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbae35bae8c047a6982e128ffb77ca86" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15e23a34059456788f9775031e27ec0"
I20240502 07:59:43.796612 20802 raft_consensus.cc:3037] T bbae35bae8c047a6982e128ffb77ca86 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:43.796595 20934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbae35bae8c047a6982e128ffb77ca86" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f471f8b5944986a7d878cf6ac550b8"
I20240502 07:59:43.796710 20934 raft_consensus.cc:3037] T bbae35bae8c047a6982e128ffb77ca86 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:43.796675 20539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbae35bae8c047a6982e128ffb77ca86" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e951f009258e473cb57cf8773c7975d4"
I20240502 07:59:43.796787 20539 raft_consensus.cc:3037] T bbae35bae8c047a6982e128ffb77ca86 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:43.796864 20273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbae35bae8c047a6982e128ffb77ca86" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7604c0eb3bf4a7cb260b564a94a1191"
I20240502 07:59:43.796988 20273 raft_consensus.cc:3037] T bbae35bae8c047a6982e128ffb77ca86 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:43.797505 20934 raft_consensus.cc:2454] T bbae35bae8c047a6982e128ffb77ca86 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b443e68b6212450286869124684bdfff in term 1.
I20240502 07:59:43.797505 20802 raft_consensus.cc:2454] T bbae35bae8c047a6982e128ffb77ca86 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b443e68b6212450286869124684bdfff in term 1.
I20240502 07:59:43.797726 20539 raft_consensus.cc:2454] T bbae35bae8c047a6982e128ffb77ca86 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b443e68b6212450286869124684bdfff in term 1.
I20240502 07:59:43.797901 20407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbae35bae8c047a6982e128ffb77ca86" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35642aa2099c40e59bc2092604078e07"
I20240502 07:59:43.798024 20407 raft_consensus.cc:3037] T bbae35bae8c047a6982e128ffb77ca86 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:43.798031 20605 leader_election.cc:302] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: b443e68b6212450286869124684bdfff, b5f471f8b5944986a7d878cf6ac550b8, c15e23a34059456788f9775031e27ec0, e951f009258e473cb57cf8773c7975d4; no voters:
I20240502 07:59:43.798051 20143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbae35bae8c047a6982e128ffb77ca86" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7180fe4574c09a8dce7c27500aab2"
I20240502 07:59:43.798151 20143 raft_consensus.cc:3037] T bbae35bae8c047a6982e128ffb77ca86 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:43.797962 20273 raft_consensus.cc:2454] T bbae35bae8c047a6982e128ffb77ca86 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b443e68b6212450286869124684bdfff in term 1.
I20240502 07:59:43.798177 21423 raft_consensus.cc:2781] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:43.798329 21423 raft_consensus.cc:686] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [term 1 LEADER]: Becoming Leader. State: Replica: b443e68b6212450286869124684bdfff, State: Running, Role: LEADER
I20240502 07:59:43.798476 21423 consensus_queue.cc:237] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:43.799000 20143 raft_consensus.cc:2454] T bbae35bae8c047a6982e128ffb77ca86 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b443e68b6212450286869124684bdfff in term 1.
I20240502 07:59:43.800526 20001 catalog_manager.cc:5484] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff reported cstate change: term changed from 0 to 1, leader changed from <none> to b443e68b6212450286869124684bdfff (127.3.139.69). New cstate: current_term: 1 leader_uuid: "b443e68b6212450286869124684bdfff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:43.811652 20407 raft_consensus.cc:2454] T bbae35bae8c047a6982e128ffb77ca86 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b443e68b6212450286869124684bdfff in term 1.
I20240502 07:59:43.883008 21785 raft_consensus.cc:482] T c3ca6f24c92d4273ac6c9d83161813d7 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:43.883145 21785 raft_consensus.cc:504] T c3ca6f24c92d4273ac6c9d83161813d7 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:43.883746 21785 leader_election.cc:288] T c3ca6f24c92d4273ac6c9d83161813d7 P e951f009258e473cb57cf8773c7975d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703), b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331), 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587), b443e68b6212450286869124684bdfff (127.3.139.69:37923), c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085), b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577)
I20240502 07:59:43.883991 20143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3ca6f24c92d4273ac6c9d83161813d7" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7180fe4574c09a8dce7c27500aab2" is_pre_election: true
I20240502 07:59:43.884179 20143 raft_consensus.cc:2379] T c3ca6f24c92d4273ac6c9d83161813d7 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e951f009258e473cb57cf8773c7975d4 in current term 1: Already voted for candidate b443e68b6212450286869124684bdfff in this term.
I20240502 07:59:43.884122 20802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3ca6f24c92d4273ac6c9d83161813d7" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15e23a34059456788f9775031e27ec0" is_pre_election: true
I20240502 07:59:43.884299 20802 raft_consensus.cc:2379] T c3ca6f24c92d4273ac6c9d83161813d7 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e951f009258e473cb57cf8773c7975d4 in current term 1: Already voted for candidate b443e68b6212450286869124684bdfff in this term.
I20240502 07:59:43.884682 20273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3ca6f24c92d4273ac6c9d83161813d7" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" is_pre_election: true
I20240502 07:59:43.884860 20273 raft_consensus.cc:2379] T c3ca6f24c92d4273ac6c9d83161813d7 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e951f009258e473cb57cf8773c7975d4 in current term 1: Already voted for candidate b443e68b6212450286869124684bdfff in this term.
I20240502 07:59:43.885284 20669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3ca6f24c92d4273ac6c9d83161813d7" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b443e68b6212450286869124684bdfff" is_pre_election: true
I20240502 07:59:43.885560 20407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3ca6f24c92d4273ac6c9d83161813d7" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35642aa2099c40e59bc2092604078e07" is_pre_election: true
I20240502 07:59:43.885610 20474 leader_election.cc:302] T c3ca6f24c92d4273ac6c9d83161813d7 P e951f009258e473cb57cf8773c7975d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: e951f009258e473cb57cf8773c7975d4; no voters: 29a7180fe4574c09a8dce7c27500aab2, b443e68b6212450286869124684bdfff, b7604c0eb3bf4a7cb260b564a94a1191, c15e23a34059456788f9775031e27ec0
I20240502 07:59:43.885739 20407 raft_consensus.cc:2379] T c3ca6f24c92d4273ac6c9d83161813d7 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e951f009258e473cb57cf8773c7975d4 in current term 1: Already voted for candidate b443e68b6212450286869124684bdfff in this term.
I20240502 07:59:43.885761 21785 raft_consensus.cc:3037] T c3ca6f24c92d4273ac6c9d83161813d7 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:43.885980 20934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3ca6f24c92d4273ac6c9d83161813d7" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f471f8b5944986a7d878cf6ac550b8" is_pre_election: true
I20240502 07:59:43.886147 20934 raft_consensus.cc:2379] T c3ca6f24c92d4273ac6c9d83161813d7 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e951f009258e473cb57cf8773c7975d4 in current term 1: Already voted for candidate b443e68b6212450286869124684bdfff in this term.
I20240502 07:59:43.886687 21785 raft_consensus.cc:2726] T c3ca6f24c92d4273ac6c9d83161813d7 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20240502 07:59:43.952396 21885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:43.952600 21885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:43.955559 21885 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_3'
I20240502 07:59:44.026994 20142 raft_consensus.cc:1261] T c3ca6f24c92d4273ac6c9d83161813d7 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:44.027429 20407 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:44.027838 21879 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:44.027920 20538 raft_consensus.cc:1261] T c3ca6f24c92d4273ac6c9d83161813d7 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:44.028236 20539 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:44.028460 21879 consensus_queue.cc:1035] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:44.028589 20801 raft_consensus.cc:1261] T c3ca6f24c92d4273ac6c9d83161813d7 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:44.028906 20802 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:44.029138 21919 consensus_queue.cc:1035] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:44.029443 20143 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:44.029793 20274 raft_consensus.cc:1261] T c3ca6f24c92d4273ac6c9d83161813d7 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:44.030089 20273 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:44.030422 20935 raft_consensus.cc:1261] T c3ca6f24c92d4273ac6c9d83161813d7 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:44.030485 21880 consensus_queue.cc:1035] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:44.030732 20934 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:44.030781 21880 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:44.031029 20405 raft_consensus.cc:1261] T c3ca6f24c92d4273ac6c9d83161813d7 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:44.031859 20273 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:44.032065 20143 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:44.032421 21509 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:44.032665 21509 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:44.034826 20669 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:44.035481 20538 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:44.035990 21882 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:44.036252 21882 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:44.037149 21929 consensus_queue.cc:1035] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:44.038110 20801 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:44.038322 20935 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:44.043226 21509 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:44.039188 21879 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:44.045114 21879 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:44.045394 21879 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:44.045636 21879 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:44.049060 21509 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:44.058404 21919 consensus_queue.cc:1035] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:44.059705 21879 consensus_queue.cc:1035] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:44.096637 21798 mvcc.cc:205] Tried to move back new op lower bound from 7023152267320442880 to 7023152266385788928. Current Snapshot: MvccSnapshot[applied={T|T < 7023152267307560960}]
I20240502 07:59:44.107307 21942 mvcc.cc:205] Tried to move back new op lower bound from 7023152267320442880 to 7023152266385788928. Current Snapshot: MvccSnapshot[applied={T|T < 7023152267307560960}]
I20240502 07:59:44.108603 21782 mvcc.cc:205] Tried to move back new op lower bound from 7023152267320442880 to 7023152266385788928. Current Snapshot: MvccSnapshot[applied={T|T < 7023152267307560960}]
I20240502 07:59:44.114264 21815 mvcc.cc:205] Tried to move back new op lower bound from 7023152267325734912 to 7023152265804632064. Current Snapshot: MvccSnapshot[applied={T|T < 7023152267308802048}]
I20240502 07:59:44.135841 21920 mvcc.cc:205] Tried to move back new op lower bound from 7023152267325734912 to 7023152265804632064. Current Snapshot: MvccSnapshot[applied={T|T < 7023152267325734912}]
I20240502 07:59:44.137370 21923 mvcc.cc:205] Tried to move back new op lower bound from 7023152267320442880 to 7023152266385788928. Current Snapshot: MvccSnapshot[applied={T|T < 7023152267320442880 or (T in {7023152267320442880})}]
INSERT report
rows total: 2000
time total: 155.907 ms
time per row: 0.0779537 ms
I20240502 07:59:44.143636 21951 mvcc.cc:205] Tried to move back new op lower bound from 7023152267325734912 to 7023152265804632064. Current Snapshot: MvccSnapshot[applied={T|T < 7023152267308802048}]
I20240502 07:59:44.162328 19996 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33518:
name: "rebalance_test_table_4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 7
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240502 07:59:44.162858 19996 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_4 in case of a server failure: 8 tablet servers would be needed, 7 are available. Consider bringing up more tablet servers.
I20240502 07:59:44.168359 20121 tablet_service.cc:1467] Processing CreateTablet for tablet 331368e52904494fa5ecf1ef594821fc (DEFAULT_TABLE table=rebalance_test_table_4 [id=8edc4db66dcd46ee8429441e0ce671a3]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:44.168561 20385 tablet_service.cc:1467] Processing CreateTablet for tablet 331368e52904494fa5ecf1ef594821fc (DEFAULT_TABLE table=rebalance_test_table_4 [id=8edc4db66dcd46ee8429441e0ce671a3]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:44.168659 20121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 331368e52904494fa5ecf1ef594821fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:44.168874 20385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 331368e52904494fa5ecf1ef594821fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:44.170868 20649 tablet_service.cc:1467] Processing CreateTablet for tablet 331368e52904494fa5ecf1ef594821fc (DEFAULT_TABLE table=rebalance_test_table_4 [id=8edc4db66dcd46ee8429441e0ce671a3]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:44.171169 20649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 331368e52904494fa5ecf1ef594821fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:44.171310 20650 tablet_service.cc:1467] Processing CreateTablet for tablet b73e0ce533014420af513429620af68c (DEFAULT_TABLE table=rebalance_test_table_4 [id=8edc4db66dcd46ee8429441e0ce671a3]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:44.171512 20650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b73e0ce533014420af513429620af68c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:44.173188 20651 tablet_service.cc:1467] Processing CreateTablet for tablet 0e2908de10684c84af542f16ff8727cb (DEFAULT_TABLE table=rebalance_test_table_4 [id=8edc4db66dcd46ee8429441e0ce671a3]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:44.173411 20651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e2908de10684c84af542f16ff8727cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:44.176283 20914 tablet_service.cc:1467] Processing CreateTablet for tablet 331368e52904494fa5ecf1ef594821fc (DEFAULT_TABLE table=rebalance_test_table_4 [id=8edc4db66dcd46ee8429441e0ce671a3]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:44.176313 20915 tablet_service.cc:1467] Processing CreateTablet for tablet b73e0ce533014420af513429620af68c (DEFAULT_TABLE table=rebalance_test_table_4 [id=8edc4db66dcd46ee8429441e0ce671a3]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:44.176542 20914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 331368e52904494fa5ecf1ef594821fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:44.176606 20913 tablet_service.cc:1467] Processing CreateTablet for tablet 0e2908de10684c84af542f16ff8727cb (DEFAULT_TABLE table=rebalance_test_table_4 [id=8edc4db66dcd46ee8429441e0ce671a3]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:44.177664 20519 tablet_service.cc:1467] Processing CreateTablet for tablet 331368e52904494fa5ecf1ef594821fc (DEFAULT_TABLE table=rebalance_test_table_4 [id=8edc4db66dcd46ee8429441e0ce671a3]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:44.177901 20519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 331368e52904494fa5ecf1ef594821fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:44.178881 20783 tablet_service.cc:1467] Processing CreateTablet for tablet 331368e52904494fa5ecf1ef594821fc (DEFAULT_TABLE table=rebalance_test_table_4 [id=8edc4db66dcd46ee8429441e0ce671a3]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:44.179157 20783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 331368e52904494fa5ecf1ef594821fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:44.179292 20518 tablet_service.cc:1467] Processing CreateTablet for tablet b73e0ce533014420af513429620af68c (DEFAULT_TABLE table=rebalance_test_table_4 [id=8edc4db66dcd46ee8429441e0ce671a3]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:44.179500 20518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b73e0ce533014420af513429620af68c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:44.180724 20517 tablet_service.cc:1467] Processing CreateTablet for tablet 0e2908de10684c84af542f16ff8727cb (DEFAULT_TABLE table=rebalance_test_table_4 [id=8edc4db66dcd46ee8429441e0ce671a3]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:44.180917 20517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e2908de10684c84af542f16ff8727cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:44.181206 20782 tablet_service.cc:1467] Processing CreateTablet for tablet b73e0ce533014420af513429620af68c (DEFAULT_TABLE table=rebalance_test_table_4 [id=8edc4db66dcd46ee8429441e0ce671a3]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:44.181404 20782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b73e0ce533014420af513429620af68c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:44.182436 20254 tablet_service.cc:1467] Processing CreateTablet for tablet 331368e52904494fa5ecf1ef594821fc (DEFAULT_TABLE table=rebalance_test_table_4 [id=8edc4db66dcd46ee8429441e0ce671a3]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:44.182693 20254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 331368e52904494fa5ecf1ef594821fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:44.183259 20781 tablet_service.cc:1467] Processing CreateTablet for tablet 0e2908de10684c84af542f16ff8727cb (DEFAULT_TABLE table=rebalance_test_table_4 [id=8edc4db66dcd46ee8429441e0ce671a3]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:44.183463 20781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e2908de10684c84af542f16ff8727cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:44.184844 20255 tablet_service.cc:1467] Processing CreateTablet for tablet b73e0ce533014420af513429620af68c (DEFAULT_TABLE table=rebalance_test_table_4 [id=8edc4db66dcd46ee8429441e0ce671a3]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:44.185052 20255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b73e0ce533014420af513429620af68c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:44.185320 20913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e2908de10684c84af542f16ff8727cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:44.186858 20253 tablet_service.cc:1467] Processing CreateTablet for tablet 0e2908de10684c84af542f16ff8727cb (DEFAULT_TABLE table=rebalance_test_table_4 [id=8edc4db66dcd46ee8429441e0ce671a3]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:44.187636 20915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b73e0ce533014420af513429620af68c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:44.188176 20387 tablet_service.cc:1467] Processing CreateTablet for tablet 0e2908de10684c84af542f16ff8727cb (DEFAULT_TABLE table=rebalance_test_table_4 [id=8edc4db66dcd46ee8429441e0ce671a3]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:44.188395 20387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e2908de10684c84af542f16ff8727cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:44.189488 21976 tablet_bootstrap.cc:492] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0: Bootstrap starting.
I20240502 07:59:44.190318 21976 tablet_bootstrap.cc:654] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:44.190548 20121 tablet_service.cc:1467] Processing CreateTablet for tablet b73e0ce533014420af513429620af68c (DEFAULT_TABLE table=rebalance_test_table_4 [id=8edc4db66dcd46ee8429441e0ce671a3]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:44.190784 20121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b73e0ce533014420af513429620af68c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:44.192818 20123 tablet_service.cc:1467] Processing CreateTablet for tablet 0e2908de10684c84af542f16ff8727cb (DEFAULT_TABLE table=rebalance_test_table_4 [id=8edc4db66dcd46ee8429441e0ce671a3]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:44.193022 20123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e2908de10684c84af542f16ff8727cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:44.195382 20253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e2908de10684c84af542f16ff8727cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:44.197058 21978 tablet_bootstrap.cc:492] T 0e2908de10684c84af542f16ff8727cb P b5f471f8b5944986a7d878cf6ac550b8: Bootstrap starting.
I20240502 07:59:44.197844 21978 tablet_bootstrap.cc:654] T 0e2908de10684c84af542f16ff8727cb P b5f471f8b5944986a7d878cf6ac550b8: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:44.201148 21978 tablet_bootstrap.cc:492] T 0e2908de10684c84af542f16ff8727cb P b5f471f8b5944986a7d878cf6ac550b8: No bootstrap required, opened a new log
I20240502 07:59:44.201221 21978 ts_tablet_manager.cc:1367] T 0e2908de10684c84af542f16ff8727cb P b5f471f8b5944986a7d878cf6ac550b8: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:59:44.201558 21978 raft_consensus.cc:348] T 0e2908de10684c84af542f16ff8727cb P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:44.201664 21978 raft_consensus.cc:374] T 0e2908de10684c84af542f16ff8727cb P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:44.201690 21978 raft_consensus.cc:729] T 0e2908de10684c84af542f16ff8727cb P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5f471f8b5944986a7d878cf6ac550b8, State: Initialized, Role: FOLLOWER
I20240502 07:59:44.201831 21978 consensus_queue.cc:260] T 0e2908de10684c84af542f16ff8727cb P b5f471f8b5944986a7d878cf6ac550b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:44.202764 20385 tablet_service.cc:1467] Processing CreateTablet for tablet b73e0ce533014420af513429620af68c (DEFAULT_TABLE table=rebalance_test_table_4 [id=8edc4db66dcd46ee8429441e0ce671a3]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:44.202992 20385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b73e0ce533014420af513429620af68c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:44.203720 21978 ts_tablet_manager.cc:1398] T 0e2908de10684c84af542f16ff8727cb P b5f471f8b5944986a7d878cf6ac550b8: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:44.205144 21978 tablet_bootstrap.cc:492] T 331368e52904494fa5ecf1ef594821fc P b5f471f8b5944986a7d878cf6ac550b8: Bootstrap starting.
I20240502 07:59:44.205674 21977 tablet_bootstrap.cc:492] T 331368e52904494fa5ecf1ef594821fc P b7604c0eb3bf4a7cb260b564a94a1191: Bootstrap starting.
I20240502 07:59:44.205854 21978 tablet_bootstrap.cc:654] T 331368e52904494fa5ecf1ef594821fc P b5f471f8b5944986a7d878cf6ac550b8: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:44.206167 21975 tablet_bootstrap.cc:492] T 331368e52904494fa5ecf1ef594821fc P b443e68b6212450286869124684bdfff: Bootstrap starting.
I20240502 07:59:44.206254 21974 tablet_bootstrap.cc:492] T 331368e52904494fa5ecf1ef594821fc P 35642aa2099c40e59bc2092604078e07: Bootstrap starting.
I20240502 07:59:44.206815 21976 tablet_bootstrap.cc:492] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0: No bootstrap required, opened a new log
I20240502 07:59:44.206912 21976 ts_tablet_manager.cc:1367] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.000s
I20240502 07:59:44.206924 21978 tablet_bootstrap.cc:492] T 331368e52904494fa5ecf1ef594821fc P b5f471f8b5944986a7d878cf6ac550b8: No bootstrap required, opened a new log
I20240502 07:59:44.206987 21978 ts_tablet_manager.cc:1367] T 331368e52904494fa5ecf1ef594821fc P b5f471f8b5944986a7d878cf6ac550b8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:44.207361 21978 raft_consensus.cc:348] T 331368e52904494fa5ecf1ef594821fc P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:44.207362 21976 raft_consensus.cc:348] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:44.207482 21978 raft_consensus.cc:374] T 331368e52904494fa5ecf1ef594821fc P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:44.207482 21976 raft_consensus.cc:374] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:44.207518 21978 raft_consensus.cc:729] T 331368e52904494fa5ecf1ef594821fc P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5f471f8b5944986a7d878cf6ac550b8, State: Initialized, Role: FOLLOWER
I20240502 07:59:44.207518 21976 raft_consensus.cc:729] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c15e23a34059456788f9775031e27ec0, State: Initialized, Role: FOLLOWER
I20240502 07:59:44.207655 21976 consensus_queue.cc:260] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:44.207655 21978 consensus_queue.cc:260] T 331368e52904494fa5ecf1ef594821fc P b5f471f8b5944986a7d878cf6ac550b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:44.208009 21976 ts_tablet_manager.cc:1398] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:44.208169 21976 tablet_bootstrap.cc:492] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0: Bootstrap starting.
I20240502 07:59:44.208572 21978 ts_tablet_manager.cc:1398] T 331368e52904494fa5ecf1ef594821fc P b5f471f8b5944986a7d878cf6ac550b8: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:44.208735 21978 tablet_bootstrap.cc:492] T b73e0ce533014420af513429620af68c P b5f471f8b5944986a7d878cf6ac550b8: Bootstrap starting.
I20240502 07:59:44.208884 21976 tablet_bootstrap.cc:654] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:44.209465 21977 tablet_bootstrap.cc:654] T 331368e52904494fa5ecf1ef594821fc P b7604c0eb3bf4a7cb260b564a94a1191: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:44.209951 21975 tablet_bootstrap.cc:654] T 331368e52904494fa5ecf1ef594821fc P b443e68b6212450286869124684bdfff: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:44.210708 21978 tablet_bootstrap.cc:654] T b73e0ce533014420af513429620af68c P b5f471f8b5944986a7d878cf6ac550b8: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:44.210767 21981 tablet_bootstrap.cc:492] T b73e0ce533014420af513429620af68c P e951f009258e473cb57cf8773c7975d4: Bootstrap starting.
I20240502 07:59:44.211623 21981 tablet_bootstrap.cc:654] T b73e0ce533014420af513429620af68c P e951f009258e473cb57cf8773c7975d4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:44.211870 21978 tablet_bootstrap.cc:492] T b73e0ce533014420af513429620af68c P b5f471f8b5944986a7d878cf6ac550b8: No bootstrap required, opened a new log
I20240502 07:59:44.211944 21978 ts_tablet_manager.cc:1367] T b73e0ce533014420af513429620af68c P b5f471f8b5944986a7d878cf6ac550b8: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:59:44.212080 21974 tablet_bootstrap.cc:654] T 331368e52904494fa5ecf1ef594821fc P 35642aa2099c40e59bc2092604078e07: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:44.212291 21978 raft_consensus.cc:348] T b73e0ce533014420af513429620af68c P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.212399 21978 raft_consensus.cc:374] T b73e0ce533014420af513429620af68c P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:44.212430 21978 raft_consensus.cc:729] T b73e0ce533014420af513429620af68c P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5f471f8b5944986a7d878cf6ac550b8, State: Initialized, Role: FOLLOWER
I20240502 07:59:44.212536 21978 consensus_queue.cc:260] T b73e0ce533014420af513429620af68c P b5f471f8b5944986a7d878cf6ac550b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.212803 21978 ts_tablet_manager.cc:1398] T b73e0ce533014420af513429620af68c P b5f471f8b5944986a7d878cf6ac550b8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:44.213021 21976 tablet_bootstrap.cc:492] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0: No bootstrap required, opened a new log
I20240502 07:59:44.213104 21976 ts_tablet_manager.cc:1367] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 07:59:44.213506 21976 raft_consensus.cc:348] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.213619 21976 raft_consensus.cc:374] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:44.213644 21976 raft_consensus.cc:729] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c15e23a34059456788f9775031e27ec0, State: Initialized, Role: FOLLOWER
I20240502 07:59:44.213778 21976 consensus_queue.cc:260] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.214118 21976 ts_tablet_manager.cc:1398] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:44.214252 21976 tablet_bootstrap.cc:492] T 0e2908de10684c84af542f16ff8727cb P c15e23a34059456788f9775031e27ec0: Bootstrap starting.
I20240502 07:59:44.214862 21801 raft_consensus.cc:482] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:44.214936 21801 raft_consensus.cc:504] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:44.215545 21973 tablet_bootstrap.cc:492] T 331368e52904494fa5ecf1ef594821fc P 29a7180fe4574c09a8dce7c27500aab2: Bootstrap starting.
I20240502 07:59:44.216306 21801 leader_election.cc:288] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703), b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331), 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587), b443e68b6212450286869124684bdfff (127.3.139.69:37923), e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315), b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577)
I20240502 07:59:44.216547 21973 tablet_bootstrap.cc:654] T 331368e52904494fa5ecf1ef594821fc P 29a7180fe4574c09a8dce7c27500aab2: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:44.217749 21977 tablet_bootstrap.cc:492] T 331368e52904494fa5ecf1ef594821fc P b7604c0eb3bf4a7cb260b564a94a1191: No bootstrap required, opened a new log
I20240502 07:59:44.217839 21977 ts_tablet_manager.cc:1367] T 331368e52904494fa5ecf1ef594821fc P b7604c0eb3bf4a7cb260b564a94a1191: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20240502 07:59:44.218186 21977 raft_consensus.cc:348] T 331368e52904494fa5ecf1ef594821fc P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:44.218304 21977 raft_consensus.cc:374] T 331368e52904494fa5ecf1ef594821fc P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:44.218329 21977 raft_consensus.cc:729] T 331368e52904494fa5ecf1ef594821fc P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7604c0eb3bf4a7cb260b564a94a1191, State: Initialized, Role: FOLLOWER
I20240502 07:59:44.218458 21977 consensus_queue.cc:260] T 331368e52904494fa5ecf1ef594821fc P b7604c0eb3bf4a7cb260b564a94a1191 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:44.218616 21981 tablet_bootstrap.cc:492] T b73e0ce533014420af513429620af68c P e951f009258e473cb57cf8773c7975d4: No bootstrap required, opened a new log
I20240502 07:59:44.218701 21981 ts_tablet_manager.cc:1367] T b73e0ce533014420af513429620af68c P e951f009258e473cb57cf8773c7975d4: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240502 07:59:44.218772 21977 ts_tablet_manager.cc:1398] T 331368e52904494fa5ecf1ef594821fc P b7604c0eb3bf4a7cb260b564a94a1191: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:44.218931 21977 tablet_bootstrap.cc:492] T b73e0ce533014420af513429620af68c P b7604c0eb3bf4a7cb260b564a94a1191: Bootstrap starting.
I20240502 07:59:44.219116 21981 raft_consensus.cc:348] T b73e0ce533014420af513429620af68c P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.219242 21981 raft_consensus.cc:374] T b73e0ce533014420af513429620af68c P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:44.219270 21981 raft_consensus.cc:729] T b73e0ce533014420af513429620af68c P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e951f009258e473cb57cf8773c7975d4, State: Initialized, Role: FOLLOWER
I20240502 07:59:44.219388 21981 consensus_queue.cc:260] T b73e0ce533014420af513429620af68c P e951f009258e473cb57cf8773c7975d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.219739 21977 tablet_bootstrap.cc:654] T b73e0ce533014420af513429620af68c P b7604c0eb3bf4a7cb260b564a94a1191: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:44.219755 21981 ts_tablet_manager.cc:1398] T b73e0ce533014420af513429620af68c P e951f009258e473cb57cf8773c7975d4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:44.219892 21981 tablet_bootstrap.cc:492] T 331368e52904494fa5ecf1ef594821fc P e951f009258e473cb57cf8773c7975d4: Bootstrap starting.
I20240502 07:59:44.220605 21981 tablet_bootstrap.cc:654] T 331368e52904494fa5ecf1ef594821fc P e951f009258e473cb57cf8773c7975d4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:44.217999 21974 tablet_bootstrap.cc:492] T 331368e52904494fa5ecf1ef594821fc P 35642aa2099c40e59bc2092604078e07: No bootstrap required, opened a new log
I20240502 07:59:44.221743 21974 ts_tablet_manager.cc:1367] T 331368e52904494fa5ecf1ef594821fc P 35642aa2099c40e59bc2092604078e07: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20240502 07:59:44.222175 21974 raft_consensus.cc:348] T 331368e52904494fa5ecf1ef594821fc P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:44.222347 21974 raft_consensus.cc:374] T 331368e52904494fa5ecf1ef594821fc P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:44.222409 21974 raft_consensus.cc:729] T 331368e52904494fa5ecf1ef594821fc P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35642aa2099c40e59bc2092604078e07, State: Initialized, Role: FOLLOWER
I20240502 07:59:44.222584 21974 consensus_queue.cc:260] T 331368e52904494fa5ecf1ef594821fc P 35642aa2099c40e59bc2092604078e07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:44.222990 21974 ts_tablet_manager.cc:1398] T 331368e52904494fa5ecf1ef594821fc P 35642aa2099c40e59bc2092604078e07: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:44.223433 21974 tablet_bootstrap.cc:492] T 0e2908de10684c84af542f16ff8727cb P 35642aa2099c40e59bc2092604078e07: Bootstrap starting.
I20240502 07:59:44.224236 21974 tablet_bootstrap.cc:654] T 0e2908de10684c84af542f16ff8727cb P 35642aa2099c40e59bc2092604078e07: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:44.225095 21975 tablet_bootstrap.cc:492] T 331368e52904494fa5ecf1ef594821fc P b443e68b6212450286869124684bdfff: No bootstrap required, opened a new log
I20240502 07:59:44.225224 21975 ts_tablet_manager.cc:1367] T 331368e52904494fa5ecf1ef594821fc P b443e68b6212450286869124684bdfff: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.000s
I20240502 07:59:44.225648 21975 raft_consensus.cc:348] T 331368e52904494fa5ecf1ef594821fc P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:44.225775 21975 raft_consensus.cc:374] T 331368e52904494fa5ecf1ef594821fc P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:44.225800 21975 raft_consensus.cc:729] T 331368e52904494fa5ecf1ef594821fc P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b443e68b6212450286869124684bdfff, State: Initialized, Role: FOLLOWER
I20240502 07:59:44.225930 21975 consensus_queue.cc:260] T 331368e52904494fa5ecf1ef594821fc P b443e68b6212450286869124684bdfff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:44.226920 21975 ts_tablet_manager.cc:1398] T 331368e52904494fa5ecf1ef594821fc P b443e68b6212450286869124684bdfff: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:44.227103 21975 tablet_bootstrap.cc:492] T b73e0ce533014420af513429620af68c P b443e68b6212450286869124684bdfff: Bootstrap starting.
I20240502 07:59:44.228281 21975 tablet_bootstrap.cc:654] T b73e0ce533014420af513429620af68c P b443e68b6212450286869124684bdfff: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:44.229562 21973 tablet_bootstrap.cc:492] T 331368e52904494fa5ecf1ef594821fc P 29a7180fe4574c09a8dce7c27500aab2: No bootstrap required, opened a new log
I20240502 07:59:44.229679 21973 ts_tablet_manager.cc:1367] T 331368e52904494fa5ecf1ef594821fc P 29a7180fe4574c09a8dce7c27500aab2: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.000s
I20240502 07:59:44.230074 21981 tablet_bootstrap.cc:492] T 331368e52904494fa5ecf1ef594821fc P e951f009258e473cb57cf8773c7975d4: No bootstrap required, opened a new log
I20240502 07:59:44.230077 21973 raft_consensus.cc:348] T 331368e52904494fa5ecf1ef594821fc P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:44.230171 21981 ts_tablet_manager.cc:1367] T 331368e52904494fa5ecf1ef594821fc P e951f009258e473cb57cf8773c7975d4: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20240502 07:59:44.230206 21973 raft_consensus.cc:374] T 331368e52904494fa5ecf1ef594821fc P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:44.230243 21973 raft_consensus.cc:729] T 331368e52904494fa5ecf1ef594821fc P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29a7180fe4574c09a8dce7c27500aab2, State: Initialized, Role: FOLLOWER
I20240502 07:59:44.230388 21973 consensus_queue.cc:260] T 331368e52904494fa5ecf1ef594821fc P 29a7180fe4574c09a8dce7c27500aab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:44.230751 21973 ts_tablet_manager.cc:1398] T 331368e52904494fa5ecf1ef594821fc P 29a7180fe4574c09a8dce7c27500aab2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:44.230705 21981 raft_consensus.cc:348] T 331368e52904494fa5ecf1ef594821fc P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:44.230810 21981 raft_consensus.cc:374] T 331368e52904494fa5ecf1ef594821fc P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:44.230840 21981 raft_consensus.cc:729] T 331368e52904494fa5ecf1ef594821fc P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e951f009258e473cb57cf8773c7975d4, State: Initialized, Role: FOLLOWER
I20240502 07:59:44.230903 21973 tablet_bootstrap.cc:492] T b73e0ce533014420af513429620af68c P 29a7180fe4574c09a8dce7c27500aab2: Bootstrap starting.
I20240502 07:59:44.230945 21981 consensus_queue.cc:260] T 331368e52904494fa5ecf1ef594821fc P e951f009258e473cb57cf8773c7975d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:44.231333 21981 ts_tablet_manager.cc:1398] T 331368e52904494fa5ecf1ef594821fc P e951f009258e473cb57cf8773c7975d4: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:44.231498 21981 tablet_bootstrap.cc:492] T 0e2908de10684c84af542f16ff8727cb P e951f009258e473cb57cf8773c7975d4: Bootstrap starting.
I20240502 07:59:44.232223 21975 tablet_bootstrap.cc:492] T b73e0ce533014420af513429620af68c P b443e68b6212450286869124684bdfff: No bootstrap required, opened a new log
I20240502 07:59:44.232304 21981 tablet_bootstrap.cc:654] T 0e2908de10684c84af542f16ff8727cb P e951f009258e473cb57cf8773c7975d4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:44.232304 21975 ts_tablet_manager.cc:1367] T b73e0ce533014420af513429620af68c P b443e68b6212450286869124684bdfff: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 07:59:44.232700 21975 raft_consensus.cc:348] T b73e0ce533014420af513429620af68c P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.232826 21975 raft_consensus.cc:374] T b73e0ce533014420af513429620af68c P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:44.232856 21975 raft_consensus.cc:729] T b73e0ce533014420af513429620af68c P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b443e68b6212450286869124684bdfff, State: Initialized, Role: FOLLOWER
I20240502 07:59:44.233265 21974 tablet_bootstrap.cc:492] T 0e2908de10684c84af542f16ff8727cb P 35642aa2099c40e59bc2092604078e07: No bootstrap required, opened a new log
I20240502 07:59:44.233286 21981 tablet_bootstrap.cc:492] T 0e2908de10684c84af542f16ff8727cb P e951f009258e473cb57cf8773c7975d4: No bootstrap required, opened a new log
I20240502 07:59:44.233345 21974 ts_tablet_manager.cc:1367] T 0e2908de10684c84af542f16ff8727cb P 35642aa2099c40e59bc2092604078e07: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20240502 07:59:44.233361 21981 ts_tablet_manager.cc:1367] T 0e2908de10684c84af542f16ff8727cb P e951f009258e473cb57cf8773c7975d4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240502 07:59:44.233731 21981 raft_consensus.cc:348] T 0e2908de10684c84af542f16ff8727cb P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:44.233728 21974 raft_consensus.cc:348] T 0e2908de10684c84af542f16ff8727cb P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:44.233819 21981 raft_consensus.cc:374] T 0e2908de10684c84af542f16ff8727cb P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:44.233839 21981 raft_consensus.cc:729] T 0e2908de10684c84af542f16ff8727cb P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e951f009258e473cb57cf8773c7975d4, State: Initialized, Role: FOLLOWER
I20240502 07:59:44.233839 21974 raft_consensus.cc:374] T 0e2908de10684c84af542f16ff8727cb P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:44.233870 21974 raft_consensus.cc:729] T 0e2908de10684c84af542f16ff8727cb P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35642aa2099c40e59bc2092604078e07, State: Initialized, Role: FOLLOWER
I20240502 07:59:44.233933 21981 consensus_queue.cc:260] T 0e2908de10684c84af542f16ff8727cb P e951f009258e473cb57cf8773c7975d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:44.233989 21974 consensus_queue.cc:260] T 0e2908de10684c84af542f16ff8727cb P 35642aa2099c40e59bc2092604078e07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:44.234198 21981 ts_tablet_manager.cc:1398] T 0e2908de10684c84af542f16ff8727cb P e951f009258e473cb57cf8773c7975d4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:44.234259 21974 ts_tablet_manager.cc:1398] T 0e2908de10684c84af542f16ff8727cb P 35642aa2099c40e59bc2092604078e07: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:44.234411 21975 consensus_queue.cc:260] T b73e0ce533014420af513429620af68c P b443e68b6212450286869124684bdfff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.234694 21975 ts_tablet_manager.cc:1398] T b73e0ce533014420af513429620af68c P b443e68b6212450286869124684bdfff: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:44.235489 21973 tablet_bootstrap.cc:654] T b73e0ce533014420af513429620af68c P 29a7180fe4574c09a8dce7c27500aab2: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:44.236392 21977 tablet_bootstrap.cc:492] T b73e0ce533014420af513429620af68c P b7604c0eb3bf4a7cb260b564a94a1191: No bootstrap required, opened a new log
I20240502 07:59:44.236480 21977 ts_tablet_manager.cc:1367] T b73e0ce533014420af513429620af68c P b7604c0eb3bf4a7cb260b564a94a1191: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.000s
I20240502 07:59:44.237068 21977 raft_consensus.cc:348] T b73e0ce533014420af513429620af68c P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.237213 21977 raft_consensus.cc:374] T b73e0ce533014420af513429620af68c P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:44.237252 21977 raft_consensus.cc:729] T b73e0ce533014420af513429620af68c P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7604c0eb3bf4a7cb260b564a94a1191, State: Initialized, Role: FOLLOWER
I20240502 07:59:44.237406 21977 consensus_queue.cc:260] T b73e0ce533014420af513429620af68c P b7604c0eb3bf4a7cb260b564a94a1191 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.237766 21977 ts_tablet_manager.cc:1398] T b73e0ce533014420af513429620af68c P b7604c0eb3bf4a7cb260b564a94a1191: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:44.237912 21975 tablet_bootstrap.cc:492] T 0e2908de10684c84af542f16ff8727cb P b443e68b6212450286869124684bdfff: Bootstrap starting.
I20240502 07:59:44.237927 21977 tablet_bootstrap.cc:492] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191: Bootstrap starting.
I20240502 07:59:44.238996 21974 tablet_bootstrap.cc:492] T b73e0ce533014420af513429620af68c P 35642aa2099c40e59bc2092604078e07: Bootstrap starting.
I20240502 07:59:44.239122 21977 tablet_bootstrap.cc:654] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:44.240212 20407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "331368e52904494fa5ecf1ef594821fc" candidate_uuid: "c15e23a34059456788f9775031e27ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35642aa2099c40e59bc2092604078e07" is_pre_election: true
I20240502 07:59:44.240320 20407 raft_consensus.cc:2454] T 331368e52904494fa5ecf1ef594821fc P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c15e23a34059456788f9775031e27ec0 in term 0.
I20240502 07:59:44.240824 21976 tablet_bootstrap.cc:654] T 0e2908de10684c84af542f16ff8727cb P c15e23a34059456788f9775031e27ec0: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:44.240942 21974 tablet_bootstrap.cc:654] T b73e0ce533014420af513429620af68c P 35642aa2099c40e59bc2092604078e07: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:44.238782 21975 tablet_bootstrap.cc:654] T 0e2908de10684c84af542f16ff8727cb P b443e68b6212450286869124684bdfff: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:44.242841 21975 tablet_bootstrap.cc:492] T 0e2908de10684c84af542f16ff8727cb P b443e68b6212450286869124684bdfff: No bootstrap required, opened a new log
I20240502 07:59:44.242993 21975 ts_tablet_manager.cc:1367] T 0e2908de10684c84af542f16ff8727cb P b443e68b6212450286869124684bdfff: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:59:44.243285 21801 raft_consensus.cc:482] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:44.243372 21801 raft_consensus.cc:504] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.244678 21974 tablet_bootstrap.cc:492] T b73e0ce533014420af513429620af68c P 35642aa2099c40e59bc2092604078e07: No bootstrap required, opened a new log
I20240502 07:59:44.244769 21974 ts_tablet_manager.cc:1367] T b73e0ce533014420af513429620af68c P 35642aa2099c40e59bc2092604078e07: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20240502 07:59:44.245229 21974 raft_consensus.cc:348] T b73e0ce533014420af513429620af68c P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.245363 21974 raft_consensus.cc:374] T b73e0ce533014420af513429620af68c P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:44.245406 21974 raft_consensus.cc:729] T b73e0ce533014420af513429620af68c P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35642aa2099c40e59bc2092604078e07, State: Initialized, Role: FOLLOWER
I20240502 07:59:44.245520 20933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "331368e52904494fa5ecf1ef594821fc" candidate_uuid: "c15e23a34059456788f9775031e27ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f471f8b5944986a7d878cf6ac550b8" is_pre_election: true
I20240502 07:59:44.245558 21974 consensus_queue.cc:260] T b73e0ce533014420af513429620af68c P 35642aa2099c40e59bc2092604078e07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.245679 20933 raft_consensus.cc:2454] T 331368e52904494fa5ecf1ef594821fc P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c15e23a34059456788f9775031e27ec0 in term 0.
I20240502 07:59:44.245905 20934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b73e0ce533014420af513429620af68c" candidate_uuid: "c15e23a34059456788f9775031e27ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f471f8b5944986a7d878cf6ac550b8" is_pre_election: true
I20240502 07:59:44.246021 20934 raft_consensus.cc:2454] T b73e0ce533014420af513429620af68c P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c15e23a34059456788f9775031e27ec0 in term 0.
I20240502 07:59:44.246774 21801 leader_election.cc:288] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315), b443e68b6212450286869124684bdfff (127.3.139.69:37923), b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577), b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331), 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703), 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587)
I20240502 07:59:44.247083 21973 tablet_bootstrap.cc:492] T b73e0ce533014420af513429620af68c P 29a7180fe4574c09a8dce7c27500aab2: No bootstrap required, opened a new log
I20240502 07:59:44.247131 20539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "331368e52904494fa5ecf1ef594821fc" candidate_uuid: "c15e23a34059456788f9775031e27ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e951f009258e473cb57cf8773c7975d4" is_pre_election: true
I20240502 07:59:44.247236 20539 raft_consensus.cc:2454] T 331368e52904494fa5ecf1ef594821fc P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c15e23a34059456788f9775031e27ec0 in term 0.
I20240502 07:59:44.247175 21973 ts_tablet_manager.cc:1367] T b73e0ce533014420af513429620af68c P 29a7180fe4574c09a8dce7c27500aab2: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20240502 07:59:44.247258 20407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b73e0ce533014420af513429620af68c" candidate_uuid: "c15e23a34059456788f9775031e27ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35642aa2099c40e59bc2092604078e07" is_pre_election: true
I20240502 07:59:44.247359 20407 raft_consensus.cc:2454] T b73e0ce533014420af513429620af68c P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c15e23a34059456788f9775031e27ec0 in term 0.
I20240502 07:59:44.247443 20537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b73e0ce533014420af513429620af68c" candidate_uuid: "c15e23a34059456788f9775031e27ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e951f009258e473cb57cf8773c7975d4" is_pre_election: true
I20240502 07:59:44.247545 20537 raft_consensus.cc:2454] T b73e0ce533014420af513429620af68c P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c15e23a34059456788f9775031e27ec0 in term 0.
I20240502 07:59:44.247467 21975 raft_consensus.cc:348] T 0e2908de10684c84af542f16ff8727cb P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:44.247615 21975 raft_consensus.cc:374] T 0e2908de10684c84af542f16ff8727cb P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:44.247664 21975 raft_consensus.cc:729] T 0e2908de10684c84af542f16ff8727cb P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b443e68b6212450286869124684bdfff, State: Initialized, Role: FOLLOWER
I20240502 07:59:44.247696 21973 raft_consensus.cc:348] T b73e0ce533014420af513429620af68c P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.247828 21973 raft_consensus.cc:374] T b73e0ce533014420af513429620af68c P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:44.247874 21973 raft_consensus.cc:729] T b73e0ce533014420af513429620af68c P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29a7180fe4574c09a8dce7c27500aab2, State: Initialized, Role: FOLLOWER
I20240502 07:59:44.247794 21975 consensus_queue.cc:260] T 0e2908de10684c84af542f16ff8727cb P b443e68b6212450286869124684bdfff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:44.248023 21973 consensus_queue.cc:260] T b73e0ce533014420af513429620af68c P 29a7180fe4574c09a8dce7c27500aab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.248106 20737 leader_election.cc:302] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 35642aa2099c40e59bc2092604078e07, b5f471f8b5944986a7d878cf6ac550b8, c15e23a34059456788f9775031e27ec0, e951f009258e473cb57cf8773c7975d4; no voters:
I20240502 07:59:44.248217 21975 ts_tablet_manager.cc:1398] T 0e2908de10684c84af542f16ff8727cb P b443e68b6212450286869124684bdfff: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20240502 07:59:44.248312 21973 ts_tablet_manager.cc:1398] T b73e0ce533014420af513429620af68c P 29a7180fe4574c09a8dce7c27500aab2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:44.248356 20737 leader_election.cc:302] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 35642aa2099c40e59bc2092604078e07, b5f471f8b5944986a7d878cf6ac550b8, c15e23a34059456788f9775031e27ec0, e951f009258e473cb57cf8773c7975d4; no voters:
I20240502 07:59:44.248406 21801 raft_consensus.cc:2781] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:44.248457 21801 raft_consensus.cc:482] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:44.248494 21801 raft_consensus.cc:3037] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:44.248498 21973 tablet_bootstrap.cc:492] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2: Bootstrap starting.
I20240502 07:59:44.249316 21973 tablet_bootstrap.cc:654] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:44.249464 21801 raft_consensus.cc:504] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:44.249982 21801 leader_election.cc:288] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [CANDIDATE]: Term 1 election: Requested vote from peers 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703), b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331), 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587), b443e68b6212450286869124684bdfff (127.3.139.69:37923), e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315), b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577)
I20240502 07:59:44.250067 21801 raft_consensus.cc:2781] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:44.250102 21801 raft_consensus.cc:482] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:44.250133 21801 raft_consensus.cc:3037] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:44.250269 20537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "331368e52904494fa5ecf1ef594821fc" candidate_uuid: "c15e23a34059456788f9775031e27ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e951f009258e473cb57cf8773c7975d4"
I20240502 07:59:44.250358 20537 raft_consensus.cc:3037] T 331368e52904494fa5ecf1ef594821fc P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:44.250666 20407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "331368e52904494fa5ecf1ef594821fc" candidate_uuid: "c15e23a34059456788f9775031e27ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35642aa2099c40e59bc2092604078e07"
I20240502 07:59:44.250757 20407 raft_consensus.cc:3037] T 331368e52904494fa5ecf1ef594821fc P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:44.250898 21801 raft_consensus.cc:504] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.251282 20537 raft_consensus.cc:2454] T 331368e52904494fa5ecf1ef594821fc P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c15e23a34059456788f9775031e27ec0 in term 1.
I20240502 07:59:44.251417 21801 leader_election.cc:288] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [CANDIDATE]: Term 1 election: Requested vote from peers e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315), b443e68b6212450286869124684bdfff (127.3.139.69:37923), b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577), b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331), 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703), 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587)
I20240502 07:59:44.251652 20407 raft_consensus.cc:2454] T 331368e52904494fa5ecf1ef594821fc P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c15e23a34059456788f9775031e27ec0 in term 1.
I20240502 07:59:44.252179 21974 ts_tablet_manager.cc:1398] T b73e0ce533014420af513429620af68c P 35642aa2099c40e59bc2092604078e07: Time spent starting tablet: real 0.007s user 0.000s sys 0.001s
I20240502 07:59:44.252272 20669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "331368e52904494fa5ecf1ef594821fc" candidate_uuid: "c15e23a34059456788f9775031e27ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b443e68b6212450286869124684bdfff" is_pre_election: true
I20240502 07:59:44.252378 20669 raft_consensus.cc:2454] T 331368e52904494fa5ecf1ef594821fc P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c15e23a34059456788f9775031e27ec0 in term 0.
I20240502 07:59:44.252427 20670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b73e0ce533014420af513429620af68c" candidate_uuid: "c15e23a34059456788f9775031e27ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b443e68b6212450286869124684bdfff" is_pre_election: true
I20240502 07:59:44.252507 20670 raft_consensus.cc:2454] T b73e0ce533014420af513429620af68c P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c15e23a34059456788f9775031e27ec0 in term 0.
I20240502 07:59:44.252645 20671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "331368e52904494fa5ecf1ef594821fc" candidate_uuid: "c15e23a34059456788f9775031e27ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b443e68b6212450286869124684bdfff"
I20240502 07:59:44.252738 20671 raft_consensus.cc:3037] T 331368e52904494fa5ecf1ef594821fc P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:44.253361 21976 tablet_bootstrap.cc:492] T 0e2908de10684c84af542f16ff8727cb P c15e23a34059456788f9775031e27ec0: No bootstrap required, opened a new log
I20240502 07:59:44.253455 21976 ts_tablet_manager.cc:1367] T 0e2908de10684c84af542f16ff8727cb P c15e23a34059456788f9775031e27ec0: Time spent bootstrapping tablet: real 0.039s user 0.001s sys 0.000s
I20240502 07:59:44.253672 20671 raft_consensus.cc:2454] T 331368e52904494fa5ecf1ef594821fc P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c15e23a34059456788f9775031e27ec0 in term 1.
I20240502 07:59:44.253866 20668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b73e0ce533014420af513429620af68c" candidate_uuid: "c15e23a34059456788f9775031e27ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b443e68b6212450286869124684bdfff"
I20240502 07:59:44.253887 21976 raft_consensus.cc:348] T 0e2908de10684c84af542f16ff8727cb P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:44.253962 20668 raft_consensus.cc:3037] T b73e0ce533014420af513429620af68c P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:44.254004 21976 raft_consensus.cc:374] T 0e2908de10684c84af542f16ff8727cb P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:44.254042 21976 raft_consensus.cc:729] T 0e2908de10684c84af542f16ff8727cb P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c15e23a34059456788f9775031e27ec0, State: Initialized, Role: FOLLOWER
I20240502 07:59:44.254101 20537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b73e0ce533014420af513429620af68c" candidate_uuid: "c15e23a34059456788f9775031e27ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e951f009258e473cb57cf8773c7975d4"
I20240502 07:59:44.254153 21976 consensus_queue.cc:260] T 0e2908de10684c84af542f16ff8727cb P c15e23a34059456788f9775031e27ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:44.254189 20537 raft_consensus.cc:3037] T b73e0ce533014420af513429620af68c P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:44.254436 21976 ts_tablet_manager.cc:1398] T 0e2908de10684c84af542f16ff8727cb P c15e23a34059456788f9775031e27ec0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:44.254801 20668 raft_consensus.cc:2454] T b73e0ce533014420af513429620af68c P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c15e23a34059456788f9775031e27ec0 in term 1.
I20240502 07:59:44.255040 20537 raft_consensus.cc:2454] T b73e0ce533014420af513429620af68c P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c15e23a34059456788f9775031e27ec0 in term 1.
I20240502 07:59:44.255152 21973 tablet_bootstrap.cc:492] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2: No bootstrap required, opened a new log
I20240502 07:59:44.255223 21973 ts_tablet_manager.cc:1367] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 07:59:44.255605 21973 raft_consensus.cc:348] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:44.255708 21973 raft_consensus.cc:374] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:44.255739 21973 raft_consensus.cc:729] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29a7180fe4574c09a8dce7c27500aab2, State: Initialized, Role: FOLLOWER
I20240502 07:59:44.255860 21973 consensus_queue.cc:260] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:44.256134 21973 ts_tablet_manager.cc:1398] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:44.256381 20934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "331368e52904494fa5ecf1ef594821fc" candidate_uuid: "c15e23a34059456788f9775031e27ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f471f8b5944986a7d878cf6ac550b8"
I20240502 07:59:44.256446 20933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b73e0ce533014420af513429620af68c" candidate_uuid: "c15e23a34059456788f9775031e27ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f471f8b5944986a7d878cf6ac550b8"
I20240502 07:59:44.256491 20934 raft_consensus.cc:3037] T 331368e52904494fa5ecf1ef594821fc P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:44.256510 20933 raft_consensus.cc:3037] T b73e0ce533014420af513429620af68c P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:44.257321 20933 raft_consensus.cc:2454] T b73e0ce533014420af513429620af68c P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c15e23a34059456788f9775031e27ec0 in term 1.
I20240502 07:59:44.257380 20934 raft_consensus.cc:2454] T 331368e52904494fa5ecf1ef594821fc P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c15e23a34059456788f9775031e27ec0 in term 1.
I20240502 07:59:44.257400 20142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "331368e52904494fa5ecf1ef594821fc" candidate_uuid: "c15e23a34059456788f9775031e27ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7180fe4574c09a8dce7c27500aab2" is_pre_election: true
I20240502 07:59:44.257402 20141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "331368e52904494fa5ecf1ef594821fc" candidate_uuid: "c15e23a34059456788f9775031e27ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7180fe4574c09a8dce7c27500aab2"
I20240502 07:59:44.257552 20140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b73e0ce533014420af513429620af68c" candidate_uuid: "c15e23a34059456788f9775031e27ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7180fe4574c09a8dce7c27500aab2"
I20240502 07:59:44.257691 20140 raft_consensus.cc:3037] T b73e0ce533014420af513429620af68c P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:44.257553 20143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b73e0ce533014420af513429620af68c" candidate_uuid: "c15e23a34059456788f9775031e27ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7180fe4574c09a8dce7c27500aab2" is_pre_election: true
I20240502 07:59:44.258494 20140 raft_consensus.cc:2454] T b73e0ce533014420af513429620af68c P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c15e23a34059456788f9775031e27ec0 in term 1.
I20240502 07:59:44.257512 20142 raft_consensus.cc:2454] T 331368e52904494fa5ecf1ef594821fc P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c15e23a34059456788f9775031e27ec0 in term 0.
I20240502 07:59:44.259212 20739 leader_election.cc:302] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 29a7180fe4574c09a8dce7c27500aab2, b5f471f8b5944986a7d878cf6ac550b8, c15e23a34059456788f9775031e27ec0, e951f009258e473cb57cf8773c7975d4; no voters:
I20240502 07:59:44.259326 20738 leader_election.cc:302] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 35642aa2099c40e59bc2092604078e07, b443e68b6212450286869124684bdfff, c15e23a34059456788f9775031e27ec0, e951f009258e473cb57cf8773c7975d4; no voters: 29a7180fe4574c09a8dce7c27500aab2
I20240502 07:59:44.259464 20407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b73e0ce533014420af513429620af68c" candidate_uuid: "c15e23a34059456788f9775031e27ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35642aa2099c40e59bc2092604078e07"
I20240502 07:59:44.259553 20407 raft_consensus.cc:3037] T b73e0ce533014420af513429620af68c P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:44.259553 21801 raft_consensus.cc:2781] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:44.259742 21801 raft_consensus.cc:686] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [term 1 LEADER]: Becoming Leader. State: Replica: c15e23a34059456788f9775031e27ec0, State: Running, Role: LEADER
I20240502 07:59:44.260336 20407 raft_consensus.cc:2454] T b73e0ce533014420af513429620af68c P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c15e23a34059456788f9775031e27ec0 in term 1.
I20240502 07:59:44.260003 22011 raft_consensus.cc:2781] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:44.260036 21801 consensus_queue.cc:237] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.260833 22011 raft_consensus.cc:686] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [term 1 LEADER]: Becoming Leader. State: Replica: c15e23a34059456788f9775031e27ec0, State: Running, Role: LEADER
I20240502 07:59:44.261035 22011 consensus_queue.cc:237] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:44.263285 19998 catalog_manager.cc:5484] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 reported cstate change: term changed from 0 to 1, leader changed from <none> to c15e23a34059456788f9775031e27ec0 (127.3.139.70). New cstate: current_term: 1 leader_uuid: "c15e23a34059456788f9775031e27ec0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:44.263473 21977 tablet_bootstrap.cc:492] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191: No bootstrap required, opened a new log
I20240502 07:59:44.263542 21977 ts_tablet_manager.cc:1367] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191: Time spent bootstrapping tablet: real 0.026s user 0.000s sys 0.002s
I20240502 07:59:44.263855 21977 raft_consensus.cc:348] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:44.263947 21977 raft_consensus.cc:374] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:44.263965 21977 raft_consensus.cc:729] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7604c0eb3bf4a7cb260b564a94a1191, State: Initialized, Role: FOLLOWER
I20240502 07:59:44.264065 21977 consensus_queue.cc:260] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:44.264163 20274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "331368e52904494fa5ecf1ef594821fc" candidate_uuid: "c15e23a34059456788f9775031e27ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" is_pre_election: true
I20240502 07:59:44.264246 20274 raft_consensus.cc:2454] T 331368e52904494fa5ecf1ef594821fc P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c15e23a34059456788f9775031e27ec0 in term 0.
I20240502 07:59:44.264295 20273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b73e0ce533014420af513429620af68c" candidate_uuid: "c15e23a34059456788f9775031e27ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" is_pre_election: true
I20240502 07:59:44.264413 20273 raft_consensus.cc:2454] T b73e0ce533014420af513429620af68c P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c15e23a34059456788f9775031e27ec0 in term 0.
I20240502 07:59:44.264575 20275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "331368e52904494fa5ecf1ef594821fc" candidate_uuid: "c15e23a34059456788f9775031e27ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7604c0eb3bf4a7cb260b564a94a1191"
I20240502 07:59:44.264636 20275 raft_consensus.cc:3037] T 331368e52904494fa5ecf1ef594821fc P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:44.264338 21977 ts_tablet_manager.cc:1398] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:44.265501 20275 raft_consensus.cc:2454] T 331368e52904494fa5ecf1ef594821fc P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c15e23a34059456788f9775031e27ec0 in term 1.
I20240502 07:59:44.265698 20272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b73e0ce533014420af513429620af68c" candidate_uuid: "c15e23a34059456788f9775031e27ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7604c0eb3bf4a7cb260b564a94a1191"
I20240502 07:59:44.265776 20272 raft_consensus.cc:3037] T b73e0ce533014420af513429620af68c P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:44.266546 20272 raft_consensus.cc:2454] T b73e0ce533014420af513429620af68c P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c15e23a34059456788f9775031e27ec0 in term 1.
I20240502 07:59:44.268095 19998 catalog_manager.cc:5484] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 reported cstate change: term changed from 0 to 1, leader changed from <none> to c15e23a34059456788f9775031e27ec0 (127.3.139.70). New cstate: current_term: 1 leader_uuid: "c15e23a34059456788f9775031e27ec0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:44.372610 21928 raft_consensus.cc:482] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:44.372750 21928 raft_consensus.cc:504] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:44.374241 21928 leader_election.cc:288] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703), 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587), c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085), b443e68b6212450286869124684bdfff (127.3.139.69:37923), e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315), b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577)
I20240502 07:59:44.379276 20142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e2908de10684c84af542f16ff8727cb" candidate_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7180fe4574c09a8dce7c27500aab2" is_pre_election: true
I20240502 07:59:44.379426 20142 raft_consensus.cc:2454] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7604c0eb3bf4a7cb260b564a94a1191 in term 0.
I20240502 07:59:44.380239 20668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e2908de10684c84af542f16ff8727cb" candidate_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b443e68b6212450286869124684bdfff" is_pre_election: true
I20240502 07:59:44.380367 20668 raft_consensus.cc:2454] T 0e2908de10684c84af542f16ff8727cb P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7604c0eb3bf4a7cb260b564a94a1191 in term 0.
I20240502 07:59:44.380754 20802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e2908de10684c84af542f16ff8727cb" candidate_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15e23a34059456788f9775031e27ec0" is_pre_election: true
I20240502 07:59:44.380874 20802 raft_consensus.cc:2454] T 0e2908de10684c84af542f16ff8727cb P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7604c0eb3bf4a7cb260b564a94a1191 in term 0.
I20240502 07:59:44.381152 20209 leader_election.cc:302] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 29a7180fe4574c09a8dce7c27500aab2, b443e68b6212450286869124684bdfff, b7604c0eb3bf4a7cb260b564a94a1191, c15e23a34059456788f9775031e27ec0; no voters:
I20240502 07:59:44.381366 21928 raft_consensus.cc:2781] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:44.381410 21928 raft_consensus.cc:482] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:44.381445 21928 raft_consensus.cc:3037] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:44.382453 21928 raft_consensus.cc:504] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:44.382941 20407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e2908de10684c84af542f16ff8727cb" candidate_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35642aa2099c40e59bc2092604078e07" is_pre_election: true
I20240502 07:59:44.383112 20407 raft_consensus.cc:2454] T 0e2908de10684c84af542f16ff8727cb P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7604c0eb3bf4a7cb260b564a94a1191 in term 0.
I20240502 07:59:44.383541 20668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e2908de10684c84af542f16ff8727cb" candidate_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b443e68b6212450286869124684bdfff"
I20240502 07:59:44.383646 20668 raft_consensus.cc:3037] T 0e2908de10684c84af542f16ff8727cb P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:44.383009 21928 leader_election.cc:288] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [CANDIDATE]: Term 1 election: Requested vote from peers 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703), 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587), c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085), b443e68b6212450286869124684bdfff (127.3.139.69:37923), e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315), b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577)
I20240502 07:59:44.384263 20142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e2908de10684c84af542f16ff8727cb" candidate_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7180fe4574c09a8dce7c27500aab2"
I20240502 07:59:44.384378 20142 raft_consensus.cc:3037] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:44.384351 20933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e2908de10684c84af542f16ff8727cb" candidate_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f471f8b5944986a7d878cf6ac550b8"
I20240502 07:59:44.384456 20933 raft_consensus.cc:3037] T 0e2908de10684c84af542f16ff8727cb P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:44.384565 20668 raft_consensus.cc:2454] T 0e2908de10684c84af542f16ff8727cb P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7604c0eb3bf4a7cb260b564a94a1191 in term 1.
I20240502 07:59:44.384778 20934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e2908de10684c84af542f16ff8727cb" candidate_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f471f8b5944986a7d878cf6ac550b8" is_pre_election: true
I20240502 07:59:44.385265 20142 raft_consensus.cc:2454] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7604c0eb3bf4a7cb260b564a94a1191 in term 1.
I20240502 07:59:44.385265 20933 raft_consensus.cc:2454] T 0e2908de10684c84af542f16ff8727cb P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7604c0eb3bf4a7cb260b564a94a1191 in term 1.
I20240502 07:59:44.385433 20802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e2908de10684c84af542f16ff8727cb" candidate_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15e23a34059456788f9775031e27ec0"
I20240502 07:59:44.385551 20802 raft_consensus.cc:3037] T 0e2908de10684c84af542f16ff8727cb P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:44.385583 20407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e2908de10684c84af542f16ff8727cb" candidate_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35642aa2099c40e59bc2092604078e07"
I20240502 07:59:44.385691 20407 raft_consensus.cc:3037] T 0e2908de10684c84af542f16ff8727cb P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:44.386458 20802 raft_consensus.cc:2454] T 0e2908de10684c84af542f16ff8727cb P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7604c0eb3bf4a7cb260b564a94a1191 in term 1.
I20240502 07:59:44.386545 20407 raft_consensus.cc:2454] T 0e2908de10684c84af542f16ff8727cb P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7604c0eb3bf4a7cb260b564a94a1191 in term 1.
I20240502 07:59:44.386731 20209 leader_election.cc:302] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 29a7180fe4574c09a8dce7c27500aab2, b443e68b6212450286869124684bdfff, b7604c0eb3bf4a7cb260b564a94a1191, c15e23a34059456788f9775031e27ec0; no voters:
I20240502 07:59:44.386749 20537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e2908de10684c84af542f16ff8727cb" candidate_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e951f009258e473cb57cf8773c7975d4" is_pre_election: true
I20240502 07:59:44.386873 20537 raft_consensus.cc:2454] T 0e2908de10684c84af542f16ff8727cb P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7604c0eb3bf4a7cb260b564a94a1191 in term 0.
I20240502 07:59:44.386878 21928 raft_consensus.cc:2781] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:44.386948 21928 raft_consensus.cc:686] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 LEADER]: Becoming Leader. State: Replica: b7604c0eb3bf4a7cb260b564a94a1191, State: Running, Role: LEADER
I20240502 07:59:44.387035 20539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e2908de10684c84af542f16ff8727cb" candidate_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e951f009258e473cb57cf8773c7975d4"
I20240502 07:59:44.387141 20539 raft_consensus.cc:3037] T 0e2908de10684c84af542f16ff8727cb P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:44.387286 21928 consensus_queue.cc:237] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:44.387969 20539 raft_consensus.cc:2454] T 0e2908de10684c84af542f16ff8727cb P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7604c0eb3bf4a7cb260b564a94a1191 in term 1.
I20240502 07:59:44.390291 19998 catalog_manager.cc:5484] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 reported cstate change: term changed from 0 to 1, leader changed from <none> to b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66). New cstate: current_term: 1 leader_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: UNKNOWN } } }
W20240502 07:59:44.552348 22022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:44.552578 22022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:44.555774 22022 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_4'
I20240502 07:59:44.635690 20933 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:44.635970 20539 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:44.636526 22011 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:44.638171 20407 raft_consensus.cc:1261] T b73e0ce533014420af513429620af68c P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:44.638365 20272 raft_consensus.cc:1261] T b73e0ce533014420af513429620af68c P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:44.638655 20934 raft_consensus.cc:1261] T b73e0ce533014420af513429620af68c P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:44.638978 20668 raft_consensus.cc:1261] T b73e0ce533014420af513429620af68c P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:44.639014 20935 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:44.639292 20671 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:44.639292 20802 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:44.639468 21801 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:44.639667 20142 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:44.640954 21928 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:44.641078 20142 raft_consensus.cc:1261] T b73e0ce533014420af513429620af68c P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:44.641343 21928 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:44.641444 22009 consensus_queue.cc:1035] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:44.641654 21928 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:44.641700 21801 consensus_queue.cc:1035] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:44.641993 21801 consensus_queue.cc:1035] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:44.642256 21801 consensus_queue.cc:1035] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:44.642704 21434 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:44.642943 20539 raft_consensus.cc:1261] T b73e0ce533014420af513429620af68c P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:44.643852 20538 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:44.645505 20668 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:44.646085 20407 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:44.646512 20142 raft_consensus.cc:3037] T 331368e52904494fa5ecf1ef594821fc P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:44.646835 20405 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:44.647110 21434 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:44.647671 20142 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:44.648154 20275 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:44.651104 22009 consensus_queue.cc:1035] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:44.651307 21434 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:44.651513 22009 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:44.651800 22009 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:44.652076 22009 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:44.654232 22059 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:44.658385 22011 consensus_queue.cc:1035] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 198.259 ms
time per row: 0.0991293 ms
I20240502 07:59:44.794963 19993 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33518:
name: "rebalance_test_table_5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 7
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240502 07:59:44.795441 19993 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_5 in case of a server failure: 8 tablet servers would be needed, 7 are available. Consider bringing up more tablet servers.
I20240502 07:59:44.800140 20651 tablet_service.cc:1467] Processing CreateTablet for tablet ae6d597fbc9242539edee8685a015d99 (DEFAULT_TABLE table=rebalance_test_table_5 [id=ed17eb489c6e490e9427e387589e51fe]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:44.800433 20651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae6d597fbc9242539edee8685a015d99. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:44.800442 20915 tablet_service.cc:1467] Processing CreateTablet for tablet ae6d597fbc9242539edee8685a015d99 (DEFAULT_TABLE table=rebalance_test_table_5 [id=ed17eb489c6e490e9427e387589e51fe]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:44.800693 20915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae6d597fbc9242539edee8685a015d99. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:44.802074 20386 tablet_service.cc:1467] Processing CreateTablet for tablet 6a77bda800d44c4bbe1b082a5ea05b55 (DEFAULT_TABLE table=rebalance_test_table_5 [id=ed17eb489c6e490e9427e387589e51fe]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:44.802340 20386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a77bda800d44c4bbe1b082a5ea05b55. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:44.802604 20914 tablet_service.cc:1467] Processing CreateTablet for tablet 0f03a9b07a8c40bdbdf3ff714302a3ea (DEFAULT_TABLE table=rebalance_test_table_5 [id=ed17eb489c6e490e9427e387589e51fe]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:44.802809 20914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f03a9b07a8c40bdbdf3ff714302a3ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:44.802839 22118 tablet_bootstrap.cc:492] T ae6d597fbc9242539edee8685a015d99 P b5f471f8b5944986a7d878cf6ac550b8: Bootstrap starting.
I20240502 07:59:44.802841 20517 tablet_service.cc:1467] Processing CreateTablet for tablet ae6d597fbc9242539edee8685a015d99 (DEFAULT_TABLE table=rebalance_test_table_5 [id=ed17eb489c6e490e9427e387589e51fe]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:44.803113 20517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae6d597fbc9242539edee8685a015d99. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:44.803305 20913 tablet_service.cc:1467] Processing CreateTablet for tablet 6a77bda800d44c4bbe1b082a5ea05b55 (DEFAULT_TABLE table=rebalance_test_table_5 [id=ed17eb489c6e490e9427e387589e51fe]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:44.803509 20913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a77bda800d44c4bbe1b082a5ea05b55. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:44.803668 22118 tablet_bootstrap.cc:654] T ae6d597fbc9242539edee8685a015d99 P b5f471f8b5944986a7d878cf6ac550b8: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:44.804396 22120 tablet_bootstrap.cc:492] T 6a77bda800d44c4bbe1b082a5ea05b55 P 35642aa2099c40e59bc2092604078e07: Bootstrap starting.
I20240502 07:59:44.805181 22120 tablet_bootstrap.cc:654] T 6a77bda800d44c4bbe1b082a5ea05b55 P 35642aa2099c40e59bc2092604078e07: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:44.805431 20385 tablet_service.cc:1467] Processing CreateTablet for tablet ae6d597fbc9242539edee8685a015d99 (DEFAULT_TABLE table=rebalance_test_table_5 [id=ed17eb489c6e490e9427e387589e51fe]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:44.805655 20385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae6d597fbc9242539edee8685a015d99. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:44.805627 20650 tablet_service.cc:1467] Processing CreateTablet for tablet 6a77bda800d44c4bbe1b082a5ea05b55 (DEFAULT_TABLE table=rebalance_test_table_5 [id=ed17eb489c6e490e9427e387589e51fe]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:44.805864 20650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a77bda800d44c4bbe1b082a5ea05b55. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:44.806021 22120 tablet_bootstrap.cc:492] T 6a77bda800d44c4bbe1b082a5ea05b55 P 35642aa2099c40e59bc2092604078e07: No bootstrap required, opened a new log
I20240502 07:59:44.806102 22120 ts_tablet_manager.cc:1367] T 6a77bda800d44c4bbe1b082a5ea05b55 P 35642aa2099c40e59bc2092604078e07: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:44.806502 22120 raft_consensus.cc:348] T 6a77bda800d44c4bbe1b082a5ea05b55 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.806610 22120 raft_consensus.cc:374] T 6a77bda800d44c4bbe1b082a5ea05b55 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:44.806643 22120 raft_consensus.cc:729] T 6a77bda800d44c4bbe1b082a5ea05b55 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35642aa2099c40e59bc2092604078e07, State: Initialized, Role: FOLLOWER
I20240502 07:59:44.806757 22120 consensus_queue.cc:260] T 6a77bda800d44c4bbe1b082a5ea05b55 P 35642aa2099c40e59bc2092604078e07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.807029 22120 ts_tablet_manager.cc:1398] T 6a77bda800d44c4bbe1b082a5ea05b55 P 35642aa2099c40e59bc2092604078e07: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:44.807624 20387 tablet_service.cc:1467] Processing CreateTablet for tablet 0f03a9b07a8c40bdbdf3ff714302a3ea (DEFAULT_TABLE table=rebalance_test_table_5 [id=ed17eb489c6e490e9427e387589e51fe]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:44.807832 20387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f03a9b07a8c40bdbdf3ff714302a3ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:44.808282 20651 tablet_service.cc:1467] Processing CreateTablet for tablet 0f03a9b07a8c40bdbdf3ff714302a3ea (DEFAULT_TABLE table=rebalance_test_table_5 [id=ed17eb489c6e490e9427e387589e51fe]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:44.808506 20651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f03a9b07a8c40bdbdf3ff714302a3ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:44.809793 20253 tablet_service.cc:1467] Processing CreateTablet for tablet ae6d597fbc9242539edee8685a015d99 (DEFAULT_TABLE table=rebalance_test_table_5 [id=ed17eb489c6e490e9427e387589e51fe]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:44.810043 20253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae6d597fbc9242539edee8685a015d99. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:44.810425 22121 tablet_bootstrap.cc:492] T ae6d597fbc9242539edee8685a015d99 P e951f009258e473cb57cf8773c7975d4: Bootstrap starting.
I20240502 07:59:44.811208 22121 tablet_bootstrap.cc:654] T ae6d597fbc9242539edee8685a015d99 P e951f009258e473cb57cf8773c7975d4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:44.811267 20255 tablet_service.cc:1467] Processing CreateTablet for tablet 0f03a9b07a8c40bdbdf3ff714302a3ea (DEFAULT_TABLE table=rebalance_test_table_5 [id=ed17eb489c6e490e9427e387589e51fe]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:44.811468 20255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f03a9b07a8c40bdbdf3ff714302a3ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:44.812142 22118 tablet_bootstrap.cc:492] T ae6d597fbc9242539edee8685a015d99 P b5f471f8b5944986a7d878cf6ac550b8: No bootstrap required, opened a new log
I20240502 07:59:44.812142 22121 tablet_bootstrap.cc:492] T ae6d597fbc9242539edee8685a015d99 P e951f009258e473cb57cf8773c7975d4: No bootstrap required, opened a new log
I20240502 07:59:44.812230 22121 ts_tablet_manager.cc:1367] T ae6d597fbc9242539edee8685a015d99 P e951f009258e473cb57cf8773c7975d4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:44.812232 22118 ts_tablet_manager.cc:1367] T ae6d597fbc9242539edee8685a015d99 P b5f471f8b5944986a7d878cf6ac550b8: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20240502 07:59:44.812641 22118 raft_consensus.cc:348] T ae6d597fbc9242539edee8685a015d99 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.812638 22121 raft_consensus.cc:348] T ae6d597fbc9242539edee8685a015d99 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.812759 22118 raft_consensus.cc:374] T ae6d597fbc9242539edee8685a015d99 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:44.812760 22121 raft_consensus.cc:374] T ae6d597fbc9242539edee8685a015d99 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:44.812791 22118 raft_consensus.cc:729] T ae6d597fbc9242539edee8685a015d99 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5f471f8b5944986a7d878cf6ac550b8, State: Initialized, Role: FOLLOWER
I20240502 07:59:44.812791 22121 raft_consensus.cc:729] T ae6d597fbc9242539edee8685a015d99 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e951f009258e473cb57cf8773c7975d4, State: Initialized, Role: FOLLOWER
I20240502 07:59:44.812911 22118 consensus_queue.cc:260] T ae6d597fbc9242539edee8685a015d99 P b5f471f8b5944986a7d878cf6ac550b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.812966 22121 consensus_queue.cc:260] T ae6d597fbc9242539edee8685a015d99 P e951f009258e473cb57cf8773c7975d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.813192 22118 ts_tablet_manager.cc:1398] T ae6d597fbc9242539edee8685a015d99 P b5f471f8b5944986a7d878cf6ac550b8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:44.813269 22121 ts_tablet_manager.cc:1398] T ae6d597fbc9242539edee8685a015d99 P e951f009258e473cb57cf8773c7975d4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:44.813232 20254 tablet_service.cc:1467] Processing CreateTablet for tablet 6a77bda800d44c4bbe1b082a5ea05b55 (DEFAULT_TABLE table=rebalance_test_table_5 [id=ed17eb489c6e490e9427e387589e51fe]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:44.813347 22118 tablet_bootstrap.cc:492] T 6a77bda800d44c4bbe1b082a5ea05b55 P b5f471f8b5944986a7d878cf6ac550b8: Bootstrap starting.
I20240502 07:59:44.813441 20254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a77bda800d44c4bbe1b082a5ea05b55. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:44.814222 22118 tablet_bootstrap.cc:654] T 6a77bda800d44c4bbe1b082a5ea05b55 P b5f471f8b5944986a7d878cf6ac550b8: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:44.815145 22124 tablet_bootstrap.cc:492] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b7604c0eb3bf4a7cb260b564a94a1191: Bootstrap starting.
I20240502 07:59:44.815320 22118 tablet_bootstrap.cc:492] T 6a77bda800d44c4bbe1b082a5ea05b55 P b5f471f8b5944986a7d878cf6ac550b8: No bootstrap required, opened a new log
I20240502 07:59:44.815405 22118 ts_tablet_manager.cc:1367] T 6a77bda800d44c4bbe1b082a5ea05b55 P b5f471f8b5944986a7d878cf6ac550b8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:44.815806 22118 raft_consensus.cc:348] T 6a77bda800d44c4bbe1b082a5ea05b55 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.815923 22118 raft_consensus.cc:374] T 6a77bda800d44c4bbe1b082a5ea05b55 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:44.815956 22118 raft_consensus.cc:729] T 6a77bda800d44c4bbe1b082a5ea05b55 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5f471f8b5944986a7d878cf6ac550b8, State: Initialized, Role: FOLLOWER
I20240502 07:59:44.816061 22118 consensus_queue.cc:260] T 6a77bda800d44c4bbe1b082a5ea05b55 P b5f471f8b5944986a7d878cf6ac550b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.816327 22118 ts_tablet_manager.cc:1398] T 6a77bda800d44c4bbe1b082a5ea05b55 P b5f471f8b5944986a7d878cf6ac550b8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:44.816479 22118 tablet_bootstrap.cc:492] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b5f471f8b5944986a7d878cf6ac550b8: Bootstrap starting.
I20240502 07:59:44.816869 22124 tablet_bootstrap.cc:654] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b7604c0eb3bf4a7cb260b564a94a1191: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:44.817492 22118 tablet_bootstrap.cc:654] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b5f471f8b5944986a7d878cf6ac550b8: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:44.817488 20519 tablet_service.cc:1467] Processing CreateTablet for tablet 6a77bda800d44c4bbe1b082a5ea05b55 (DEFAULT_TABLE table=rebalance_test_table_5 [id=ed17eb489c6e490e9427e387589e51fe]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:44.817718 20519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a77bda800d44c4bbe1b082a5ea05b55. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:44.818104 22124 tablet_bootstrap.cc:492] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b7604c0eb3bf4a7cb260b564a94a1191: No bootstrap required, opened a new log
I20240502 07:59:44.818181 22124 ts_tablet_manager.cc:1367] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b7604c0eb3bf4a7cb260b564a94a1191: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:59:44.818434 22118 tablet_bootstrap.cc:492] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b5f471f8b5944986a7d878cf6ac550b8: No bootstrap required, opened a new log
I20240502 07:59:44.818511 22118 ts_tablet_manager.cc:1367] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b5f471f8b5944986a7d878cf6ac550b8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:44.818557 22124 raft_consensus.cc:348] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.818667 22124 raft_consensus.cc:374] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:44.818696 22124 raft_consensus.cc:729] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7604c0eb3bf4a7cb260b564a94a1191, State: Initialized, Role: FOLLOWER
I20240502 07:59:44.818809 22124 consensus_queue.cc:260] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b7604c0eb3bf4a7cb260b564a94a1191 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.818871 22118 raft_consensus.cc:348] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.818984 22118 raft_consensus.cc:374] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:44.819017 22118 raft_consensus.cc:729] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5f471f8b5944986a7d878cf6ac550b8, State: Initialized, Role: FOLLOWER
I20240502 07:59:44.819097 22124 ts_tablet_manager.cc:1398] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b7604c0eb3bf4a7cb260b564a94a1191: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:44.819144 22118 consensus_queue.cc:260] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b5f471f8b5944986a7d878cf6ac550b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.819252 22124 tablet_bootstrap.cc:492] T 6a77bda800d44c4bbe1b082a5ea05b55 P b7604c0eb3bf4a7cb260b564a94a1191: Bootstrap starting.
I20240502 07:59:44.819494 22118 ts_tablet_manager.cc:1398] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b5f471f8b5944986a7d878cf6ac550b8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:44.819638 20123 tablet_service.cc:1467] Processing CreateTablet for tablet ae6d597fbc9242539edee8685a015d99 (DEFAULT_TABLE table=rebalance_test_table_5 [id=ed17eb489c6e490e9427e387589e51fe]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:44.819895 20123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae6d597fbc9242539edee8685a015d99. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:44.819993 22124 tablet_bootstrap.cc:654] T 6a77bda800d44c4bbe1b082a5ea05b55 P b7604c0eb3bf4a7cb260b564a94a1191: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:44.821334 22124 tablet_bootstrap.cc:492] T 6a77bda800d44c4bbe1b082a5ea05b55 P b7604c0eb3bf4a7cb260b564a94a1191: No bootstrap required, opened a new log
I20240502 07:59:44.821417 22124 ts_tablet_manager.cc:1367] T 6a77bda800d44c4bbe1b082a5ea05b55 P b7604c0eb3bf4a7cb260b564a94a1191: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:44.821946 22131 tablet_bootstrap.cc:492] T ae6d597fbc9242539edee8685a015d99 P 29a7180fe4574c09a8dce7c27500aab2: Bootstrap starting.
I20240502 07:59:44.821923 22124 raft_consensus.cc:348] T 6a77bda800d44c4bbe1b082a5ea05b55 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.822024 22124 raft_consensus.cc:374] T 6a77bda800d44c4bbe1b082a5ea05b55 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:44.822085 20121 tablet_service.cc:1467] Processing CreateTablet for tablet 0f03a9b07a8c40bdbdf3ff714302a3ea (DEFAULT_TABLE table=rebalance_test_table_5 [id=ed17eb489c6e490e9427e387589e51fe]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:44.822299 20121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f03a9b07a8c40bdbdf3ff714302a3ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:44.822062 22124 raft_consensus.cc:729] T 6a77bda800d44c4bbe1b082a5ea05b55 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7604c0eb3bf4a7cb260b564a94a1191, State: Initialized, Role: FOLLOWER
I20240502 07:59:44.822463 22124 consensus_queue.cc:260] T 6a77bda800d44c4bbe1b082a5ea05b55 P b7604c0eb3bf4a7cb260b564a94a1191 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.822785 22124 ts_tablet_manager.cc:1398] T 6a77bda800d44c4bbe1b082a5ea05b55 P b7604c0eb3bf4a7cb260b564a94a1191: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:44.822821 22131 tablet_bootstrap.cc:654] T ae6d597fbc9242539edee8685a015d99 P 29a7180fe4574c09a8dce7c27500aab2: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:44.823319 20122 tablet_service.cc:1467] Processing CreateTablet for tablet 6a77bda800d44c4bbe1b082a5ea05b55 (DEFAULT_TABLE table=rebalance_test_table_5 [id=ed17eb489c6e490e9427e387589e51fe]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:44.823542 20122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a77bda800d44c4bbe1b082a5ea05b55. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:44.823956 22131 tablet_bootstrap.cc:492] T ae6d597fbc9242539edee8685a015d99 P 29a7180fe4574c09a8dce7c27500aab2: No bootstrap required, opened a new log
I20240502 07:59:44.824082 22131 ts_tablet_manager.cc:1367] T ae6d597fbc9242539edee8685a015d99 P 29a7180fe4574c09a8dce7c27500aab2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 07:59:44.824512 22131 raft_consensus.cc:348] T ae6d597fbc9242539edee8685a015d99 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.824625 22131 raft_consensus.cc:374] T ae6d597fbc9242539edee8685a015d99 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:44.824656 22131 raft_consensus.cc:729] T ae6d597fbc9242539edee8685a015d99 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29a7180fe4574c09a8dce7c27500aab2, State: Initialized, Role: FOLLOWER
I20240502 07:59:44.824780 22131 consensus_queue.cc:260] T ae6d597fbc9242539edee8685a015d99 P 29a7180fe4574c09a8dce7c27500aab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.825037 22131 ts_tablet_manager.cc:1398] T ae6d597fbc9242539edee8685a015d99 P 29a7180fe4574c09a8dce7c27500aab2: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:44.825256 22131 tablet_bootstrap.cc:492] T 6a77bda800d44c4bbe1b082a5ea05b55 P 29a7180fe4574c09a8dce7c27500aab2: Bootstrap starting.
I20240502 07:59:44.825559 20781 tablet_service.cc:1467] Processing CreateTablet for tablet ae6d597fbc9242539edee8685a015d99 (DEFAULT_TABLE table=rebalance_test_table_5 [id=ed17eb489c6e490e9427e387589e51fe]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:44.825559 20782 tablet_service.cc:1467] Processing CreateTablet for tablet 0f03a9b07a8c40bdbdf3ff714302a3ea (DEFAULT_TABLE table=rebalance_test_table_5 [id=ed17eb489c6e490e9427e387589e51fe]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:44.825642 20783 tablet_service.cc:1467] Processing CreateTablet for tablet 6a77bda800d44c4bbe1b082a5ea05b55 (DEFAULT_TABLE table=rebalance_test_table_5 [id=ed17eb489c6e490e9427e387589e51fe]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:44.825837 20781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae6d597fbc9242539edee8685a015d99. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:44.825973 20783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a77bda800d44c4bbe1b082a5ea05b55. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:44.827520 22120 tablet_bootstrap.cc:492] T ae6d597fbc9242539edee8685a015d99 P 35642aa2099c40e59bc2092604078e07: Bootstrap starting.
I20240502 07:59:44.827831 20782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f03a9b07a8c40bdbdf3ff714302a3ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:44.828305 22120 tablet_bootstrap.cc:654] T ae6d597fbc9242539edee8685a015d99 P 35642aa2099c40e59bc2092604078e07: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:44.829339 22131 tablet_bootstrap.cc:654] T 6a77bda800d44c4bbe1b082a5ea05b55 P 29a7180fe4574c09a8dce7c27500aab2: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:44.829900 22133 tablet_bootstrap.cc:492] T ae6d597fbc9242539edee8685a015d99 P c15e23a34059456788f9775031e27ec0: Bootstrap starting.
I20240502 07:59:44.830098 22124 tablet_bootstrap.cc:492] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191: Bootstrap starting.
I20240502 07:59:44.830688 22133 tablet_bootstrap.cc:654] T ae6d597fbc9242539edee8685a015d99 P c15e23a34059456788f9775031e27ec0: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:44.830817 22124 tablet_bootstrap.cc:654] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:44.830965 22131 tablet_bootstrap.cc:492] T 6a77bda800d44c4bbe1b082a5ea05b55 P 29a7180fe4574c09a8dce7c27500aab2: No bootstrap required, opened a new log
I20240502 07:59:44.831072 22131 ts_tablet_manager.cc:1367] T 6a77bda800d44c4bbe1b082a5ea05b55 P 29a7180fe4574c09a8dce7c27500aab2: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20240502 07:59:44.831496 22131 raft_consensus.cc:348] T 6a77bda800d44c4bbe1b082a5ea05b55 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.831573 22133 tablet_bootstrap.cc:492] T ae6d597fbc9242539edee8685a015d99 P c15e23a34059456788f9775031e27ec0: No bootstrap required, opened a new log
I20240502 07:59:44.831614 22131 raft_consensus.cc:374] T 6a77bda800d44c4bbe1b082a5ea05b55 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:44.831645 22131 raft_consensus.cc:729] T 6a77bda800d44c4bbe1b082a5ea05b55 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29a7180fe4574c09a8dce7c27500aab2, State: Initialized, Role: FOLLOWER
I20240502 07:59:44.831646 22133 ts_tablet_manager.cc:1367] T ae6d597fbc9242539edee8685a015d99 P c15e23a34059456788f9775031e27ec0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:44.831779 22131 consensus_queue.cc:260] T 6a77bda800d44c4bbe1b082a5ea05b55 P 29a7180fe4574c09a8dce7c27500aab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.832124 22131 ts_tablet_manager.cc:1398] T 6a77bda800d44c4bbe1b082a5ea05b55 P 29a7180fe4574c09a8dce7c27500aab2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:44.832194 22121 tablet_bootstrap.cc:492] T 6a77bda800d44c4bbe1b082a5ea05b55 P e951f009258e473cb57cf8773c7975d4: Bootstrap starting.
I20240502 07:59:44.832285 22131 tablet_bootstrap.cc:492] T 0f03a9b07a8c40bdbdf3ff714302a3ea P 29a7180fe4574c09a8dce7c27500aab2: Bootstrap starting.
I20240502 07:59:44.832322 22124 tablet_bootstrap.cc:492] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191: No bootstrap required, opened a new log
I20240502 07:59:44.832409 22124 ts_tablet_manager.cc:1367] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:44.832579 20517 tablet_service.cc:1467] Processing CreateTablet for tablet 0f03a9b07a8c40bdbdf3ff714302a3ea (DEFAULT_TABLE table=rebalance_test_table_5 [id=ed17eb489c6e490e9427e387589e51fe]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:44.832804 20517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f03a9b07a8c40bdbdf3ff714302a3ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:44.832782 22124 raft_consensus.cc:348] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.832880 22124 raft_consensus.cc:374] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:44.832911 22124 raft_consensus.cc:729] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7604c0eb3bf4a7cb260b564a94a1191, State: Initialized, Role: FOLLOWER
I20240502 07:59:44.832976 22121 tablet_bootstrap.cc:654] T 6a77bda800d44c4bbe1b082a5ea05b55 P e951f009258e473cb57cf8773c7975d4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:44.833037 22124 consensus_queue.cc:260] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.833106 22131 tablet_bootstrap.cc:654] T 0f03a9b07a8c40bdbdf3ff714302a3ea P 29a7180fe4574c09a8dce7c27500aab2: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:44.833307 22124 ts_tablet_manager.cc:1398] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:44.832017 22133 raft_consensus.cc:348] T ae6d597fbc9242539edee8685a015d99 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.833518 22133 raft_consensus.cc:374] T ae6d597fbc9242539edee8685a015d99 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:44.833562 22133 raft_consensus.cc:729] T ae6d597fbc9242539edee8685a015d99 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c15e23a34059456788f9775031e27ec0, State: Initialized, Role: FOLLOWER
I20240502 07:59:44.833698 22133 consensus_queue.cc:260] T ae6d597fbc9242539edee8685a015d99 P c15e23a34059456788f9775031e27ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.834039 22133 ts_tablet_manager.cc:1398] T ae6d597fbc9242539edee8685a015d99 P c15e23a34059456788f9775031e27ec0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:44.834198 22133 tablet_bootstrap.cc:492] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0: Bootstrap starting.
I20240502 07:59:44.835211 22133 tablet_bootstrap.cc:654] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:44.836122 22121 tablet_bootstrap.cc:492] T 6a77bda800d44c4bbe1b082a5ea05b55 P e951f009258e473cb57cf8773c7975d4: No bootstrap required, opened a new log
I20240502 07:59:44.836215 22121 ts_tablet_manager.cc:1367] T 6a77bda800d44c4bbe1b082a5ea05b55 P e951f009258e473cb57cf8773c7975d4: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:59:44.836614 22121 raft_consensus.cc:348] T 6a77bda800d44c4bbe1b082a5ea05b55 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.836743 22121 raft_consensus.cc:374] T 6a77bda800d44c4bbe1b082a5ea05b55 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:44.836777 22121 raft_consensus.cc:729] T 6a77bda800d44c4bbe1b082a5ea05b55 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e951f009258e473cb57cf8773c7975d4, State: Initialized, Role: FOLLOWER
I20240502 07:59:44.836874 22121 consensus_queue.cc:260] T 6a77bda800d44c4bbe1b082a5ea05b55 P e951f009258e473cb57cf8773c7975d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.837271 22131 tablet_bootstrap.cc:492] T 0f03a9b07a8c40bdbdf3ff714302a3ea P 29a7180fe4574c09a8dce7c27500aab2: No bootstrap required, opened a new log
I20240502 07:59:44.837363 22131 ts_tablet_manager.cc:1367] T 0f03a9b07a8c40bdbdf3ff714302a3ea P 29a7180fe4574c09a8dce7c27500aab2: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:59:44.837395 22121 ts_tablet_manager.cc:1398] T 6a77bda800d44c4bbe1b082a5ea05b55 P e951f009258e473cb57cf8773c7975d4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:44.837554 22121 tablet_bootstrap.cc:492] T 0f03a9b07a8c40bdbdf3ff714302a3ea P e951f009258e473cb57cf8773c7975d4: Bootstrap starting.
I20240502 07:59:44.837779 22131 raft_consensus.cc:348] T 0f03a9b07a8c40bdbdf3ff714302a3ea P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.837913 22131 raft_consensus.cc:374] T 0f03a9b07a8c40bdbdf3ff714302a3ea P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:44.837942 22117 tablet_bootstrap.cc:492] T ae6d597fbc9242539edee8685a015d99 P b443e68b6212450286869124684bdfff: Bootstrap starting.
I20240502 07:59:44.837949 22131 raft_consensus.cc:729] T 0f03a9b07a8c40bdbdf3ff714302a3ea P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29a7180fe4574c09a8dce7c27500aab2, State: Initialized, Role: FOLLOWER
I20240502 07:59:44.838058 22131 consensus_queue.cc:260] T 0f03a9b07a8c40bdbdf3ff714302a3ea P 29a7180fe4574c09a8dce7c27500aab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.838305 22121 tablet_bootstrap.cc:654] T 0f03a9b07a8c40bdbdf3ff714302a3ea P e951f009258e473cb57cf8773c7975d4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:44.838318 22131 ts_tablet_manager.cc:1398] T 0f03a9b07a8c40bdbdf3ff714302a3ea P 29a7180fe4574c09a8dce7c27500aab2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:44.838750 22117 tablet_bootstrap.cc:654] T ae6d597fbc9242539edee8685a015d99 P b443e68b6212450286869124684bdfff: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:44.839349 22121 tablet_bootstrap.cc:492] T 0f03a9b07a8c40bdbdf3ff714302a3ea P e951f009258e473cb57cf8773c7975d4: No bootstrap required, opened a new log
I20240502 07:59:44.839422 22121 ts_tablet_manager.cc:1367] T 0f03a9b07a8c40bdbdf3ff714302a3ea P e951f009258e473cb57cf8773c7975d4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:44.839820 22121 raft_consensus.cc:348] T 0f03a9b07a8c40bdbdf3ff714302a3ea P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.839913 22121 raft_consensus.cc:374] T 0f03a9b07a8c40bdbdf3ff714302a3ea P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:44.839943 22121 raft_consensus.cc:729] T 0f03a9b07a8c40bdbdf3ff714302a3ea P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e951f009258e473cb57cf8773c7975d4, State: Initialized, Role: FOLLOWER
I20240502 07:59:44.840122 22121 consensus_queue.cc:260] T 0f03a9b07a8c40bdbdf3ff714302a3ea P e951f009258e473cb57cf8773c7975d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.840368 22121 ts_tablet_manager.cc:1398] T 0f03a9b07a8c40bdbdf3ff714302a3ea P e951f009258e473cb57cf8773c7975d4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:44.840931 22120 tablet_bootstrap.cc:492] T ae6d597fbc9242539edee8685a015d99 P 35642aa2099c40e59bc2092604078e07: No bootstrap required, opened a new log
I20240502 07:59:44.840977 22133 tablet_bootstrap.cc:492] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0: No bootstrap required, opened a new log
I20240502 07:59:44.841018 22120 ts_tablet_manager.cc:1367] T ae6d597fbc9242539edee8685a015d99 P 35642aa2099c40e59bc2092604078e07: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.000s
I20240502 07:59:44.841073 22133 ts_tablet_manager.cc:1367] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240502 07:59:44.841401 22133 raft_consensus.cc:348] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.841491 22133 raft_consensus.cc:374] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:44.841519 22133 raft_consensus.cc:729] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c15e23a34059456788f9775031e27ec0, State: Initialized, Role: FOLLOWER
I20240502 07:59:44.841462 22120 raft_consensus.cc:348] T ae6d597fbc9242539edee8685a015d99 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.841574 22120 raft_consensus.cc:374] T ae6d597fbc9242539edee8685a015d99 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:44.841629 22120 raft_consensus.cc:729] T ae6d597fbc9242539edee8685a015d99 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35642aa2099c40e59bc2092604078e07, State: Initialized, Role: FOLLOWER
I20240502 07:59:44.841651 22133 consensus_queue.cc:260] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.841763 22120 consensus_queue.cc:260] T ae6d597fbc9242539edee8685a015d99 P 35642aa2099c40e59bc2092604078e07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.841977 22133 ts_tablet_manager.cc:1398] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:44.842046 22120 ts_tablet_manager.cc:1398] T ae6d597fbc9242539edee8685a015d99 P 35642aa2099c40e59bc2092604078e07: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:44.842116 22133 tablet_bootstrap.cc:492] T 0f03a9b07a8c40bdbdf3ff714302a3ea P c15e23a34059456788f9775031e27ec0: Bootstrap starting.
I20240502 07:59:44.842209 22120 tablet_bootstrap.cc:492] T 0f03a9b07a8c40bdbdf3ff714302a3ea P 35642aa2099c40e59bc2092604078e07: Bootstrap starting.
I20240502 07:59:44.842797 22133 tablet_bootstrap.cc:654] T 0f03a9b07a8c40bdbdf3ff714302a3ea P c15e23a34059456788f9775031e27ec0: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:44.843672 22133 tablet_bootstrap.cc:492] T 0f03a9b07a8c40bdbdf3ff714302a3ea P c15e23a34059456788f9775031e27ec0: No bootstrap required, opened a new log
I20240502 07:59:44.843744 22133 ts_tablet_manager.cc:1367] T 0f03a9b07a8c40bdbdf3ff714302a3ea P c15e23a34059456788f9775031e27ec0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:44.844097 22133 raft_consensus.cc:348] T 0f03a9b07a8c40bdbdf3ff714302a3ea P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.844221 22133 raft_consensus.cc:374] T 0f03a9b07a8c40bdbdf3ff714302a3ea P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:44.844255 22133 raft_consensus.cc:729] T 0f03a9b07a8c40bdbdf3ff714302a3ea P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c15e23a34059456788f9775031e27ec0, State: Initialized, Role: FOLLOWER
I20240502 07:59:44.844377 22133 consensus_queue.cc:260] T 0f03a9b07a8c40bdbdf3ff714302a3ea P c15e23a34059456788f9775031e27ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.843031 22120 tablet_bootstrap.cc:654] T 0f03a9b07a8c40bdbdf3ff714302a3ea P 35642aa2099c40e59bc2092604078e07: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:44.844667 22133 ts_tablet_manager.cc:1398] T 0f03a9b07a8c40bdbdf3ff714302a3ea P c15e23a34059456788f9775031e27ec0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:44.845988 22120 tablet_bootstrap.cc:492] T 0f03a9b07a8c40bdbdf3ff714302a3ea P 35642aa2099c40e59bc2092604078e07: No bootstrap required, opened a new log
I20240502 07:59:44.846065 22120 ts_tablet_manager.cc:1367] T 0f03a9b07a8c40bdbdf3ff714302a3ea P 35642aa2099c40e59bc2092604078e07: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:59:44.846310 22117 tablet_bootstrap.cc:492] T ae6d597fbc9242539edee8685a015d99 P b443e68b6212450286869124684bdfff: No bootstrap required, opened a new log
I20240502 07:59:44.846410 22117 ts_tablet_manager.cc:1367] T ae6d597fbc9242539edee8685a015d99 P b443e68b6212450286869124684bdfff: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20240502 07:59:44.846487 22120 raft_consensus.cc:348] T 0f03a9b07a8c40bdbdf3ff714302a3ea P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.846582 22120 raft_consensus.cc:374] T 0f03a9b07a8c40bdbdf3ff714302a3ea P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:44.846617 22120 raft_consensus.cc:729] T 0f03a9b07a8c40bdbdf3ff714302a3ea P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35642aa2099c40e59bc2092604078e07, State: Initialized, Role: FOLLOWER
I20240502 07:59:44.846724 22120 consensus_queue.cc:260] T 0f03a9b07a8c40bdbdf3ff714302a3ea P 35642aa2099c40e59bc2092604078e07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.846748 22117 raft_consensus.cc:348] T ae6d597fbc9242539edee8685a015d99 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.846834 22117 raft_consensus.cc:374] T ae6d597fbc9242539edee8685a015d99 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:44.846868 22117 raft_consensus.cc:729] T ae6d597fbc9242539edee8685a015d99 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b443e68b6212450286869124684bdfff, State: Initialized, Role: FOLLOWER
I20240502 07:59:44.846976 22120 ts_tablet_manager.cc:1398] T 0f03a9b07a8c40bdbdf3ff714302a3ea P 35642aa2099c40e59bc2092604078e07: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:44.846998 22117 consensus_queue.cc:260] T ae6d597fbc9242539edee8685a015d99 P b443e68b6212450286869124684bdfff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.847298 22117 ts_tablet_manager.cc:1398] T ae6d597fbc9242539edee8685a015d99 P b443e68b6212450286869124684bdfff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:44.847450 22117 tablet_bootstrap.cc:492] T 6a77bda800d44c4bbe1b082a5ea05b55 P b443e68b6212450286869124684bdfff: Bootstrap starting.
I20240502 07:59:44.848382 22117 tablet_bootstrap.cc:654] T 6a77bda800d44c4bbe1b082a5ea05b55 P b443e68b6212450286869124684bdfff: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:44.849520 22117 tablet_bootstrap.cc:492] T 6a77bda800d44c4bbe1b082a5ea05b55 P b443e68b6212450286869124684bdfff: No bootstrap required, opened a new log
I20240502 07:59:44.849617 22117 ts_tablet_manager.cc:1367] T 6a77bda800d44c4bbe1b082a5ea05b55 P b443e68b6212450286869124684bdfff: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:44.850010 22117 raft_consensus.cc:348] T 6a77bda800d44c4bbe1b082a5ea05b55 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.850101 22117 raft_consensus.cc:374] T 6a77bda800d44c4bbe1b082a5ea05b55 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:44.850121 22117 raft_consensus.cc:729] T 6a77bda800d44c4bbe1b082a5ea05b55 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b443e68b6212450286869124684bdfff, State: Initialized, Role: FOLLOWER
I20240502 07:59:44.850220 22117 consensus_queue.cc:260] T 6a77bda800d44c4bbe1b082a5ea05b55 P b443e68b6212450286869124684bdfff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.850456 22117 ts_tablet_manager.cc:1398] T 6a77bda800d44c4bbe1b082a5ea05b55 P b443e68b6212450286869124684bdfff: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:44.850682 22117 tablet_bootstrap.cc:492] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff: Bootstrap starting.
I20240502 07:59:44.851552 22117 tablet_bootstrap.cc:654] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:44.852613 22117 tablet_bootstrap.cc:492] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff: No bootstrap required, opened a new log
I20240502 07:59:44.852689 22117 ts_tablet_manager.cc:1367] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 07:59:44.853066 22117 raft_consensus.cc:348] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.853169 22117 raft_consensus.cc:374] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:44.853199 22117 raft_consensus.cc:729] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b443e68b6212450286869124684bdfff, State: Initialized, Role: FOLLOWER
I20240502 07:59:44.853308 22117 consensus_queue.cc:260] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.853574 22117 ts_tablet_manager.cc:1398] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:44.871613 22054 raft_consensus.cc:482] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:44.871719 22054 raft_consensus.cc:504] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.872304 22054 leader_election.cc:288] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315), b443e68b6212450286869124684bdfff (127.3.139.69:37923), b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577), b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331), 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703), 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587)
I20240502 07:59:44.872586 20275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a77bda800d44c4bbe1b082a5ea05b55" candidate_uuid: "c15e23a34059456788f9775031e27ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" is_pre_election: true
I20240502 07:59:44.872713 20275 raft_consensus.cc:2454] T 6a77bda800d44c4bbe1b082a5ea05b55 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c15e23a34059456788f9775031e27ec0 in term 0.
I20240502 07:59:44.872738 20143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a77bda800d44c4bbe1b082a5ea05b55" candidate_uuid: "c15e23a34059456788f9775031e27ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7180fe4574c09a8dce7c27500aab2" is_pre_election: true
I20240502 07:59:44.872736 20405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a77bda800d44c4bbe1b082a5ea05b55" candidate_uuid: "c15e23a34059456788f9775031e27ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35642aa2099c40e59bc2092604078e07" is_pre_election: true
I20240502 07:59:44.872859 20405 raft_consensus.cc:2454] T 6a77bda800d44c4bbe1b082a5ea05b55 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c15e23a34059456788f9775031e27ec0 in term 0.
I20240502 07:59:44.872853 20143 raft_consensus.cc:2454] T 6a77bda800d44c4bbe1b082a5ea05b55 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c15e23a34059456788f9775031e27ec0 in term 0.
I20240502 07:59:44.873205 20671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a77bda800d44c4bbe1b082a5ea05b55" candidate_uuid: "c15e23a34059456788f9775031e27ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b443e68b6212450286869124684bdfff" is_pre_election: true
I20240502 07:59:44.873276 20739 leader_election.cc:302] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 29a7180fe4574c09a8dce7c27500aab2, 35642aa2099c40e59bc2092604078e07, b7604c0eb3bf4a7cb260b564a94a1191, c15e23a34059456788f9775031e27ec0; no voters:
I20240502 07:59:44.873310 20671 raft_consensus.cc:2454] T 6a77bda800d44c4bbe1b082a5ea05b55 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c15e23a34059456788f9775031e27ec0 in term 0.
I20240502 07:59:44.873308 20934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a77bda800d44c4bbe1b082a5ea05b55" candidate_uuid: "c15e23a34059456788f9775031e27ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f471f8b5944986a7d878cf6ac550b8" is_pre_election: true
I20240502 07:59:44.873420 20934 raft_consensus.cc:2454] T 6a77bda800d44c4bbe1b082a5ea05b55 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c15e23a34059456788f9775031e27ec0 in term 0.
I20240502 07:59:44.873766 22054 raft_consensus.cc:2781] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:44.873819 22054 raft_consensus.cc:482] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:44.873809 20539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a77bda800d44c4bbe1b082a5ea05b55" candidate_uuid: "c15e23a34059456788f9775031e27ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e951f009258e473cb57cf8773c7975d4" is_pre_election: true
I20240502 07:59:44.873857 22054 raft_consensus.cc:3037] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:44.873924 20539 raft_consensus.cc:2454] T 6a77bda800d44c4bbe1b082a5ea05b55 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c15e23a34059456788f9775031e27ec0 in term 0.
I20240502 07:59:44.874714 22054 raft_consensus.cc:504] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.875241 22054 leader_election.cc:288] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [CANDIDATE]: Term 1 election: Requested vote from peers e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315), b443e68b6212450286869124684bdfff (127.3.139.69:37923), b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577), b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331), 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703), 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587)
I20240502 07:59:44.875380 20934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a77bda800d44c4bbe1b082a5ea05b55" candidate_uuid: "c15e23a34059456788f9775031e27ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f471f8b5944986a7d878cf6ac550b8"
I20240502 07:59:44.875494 20934 raft_consensus.cc:3037] T 6a77bda800d44c4bbe1b082a5ea05b55 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:44.875540 20671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a77bda800d44c4bbe1b082a5ea05b55" candidate_uuid: "c15e23a34059456788f9775031e27ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b443e68b6212450286869124684bdfff"
I20240502 07:59:44.875635 20671 raft_consensus.cc:3037] T 6a77bda800d44c4bbe1b082a5ea05b55 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:44.875681 20275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a77bda800d44c4bbe1b082a5ea05b55" candidate_uuid: "c15e23a34059456788f9775031e27ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7604c0eb3bf4a7cb260b564a94a1191"
I20240502 07:59:44.875758 20275 raft_consensus.cc:3037] T 6a77bda800d44c4bbe1b082a5ea05b55 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:44.876343 20934 raft_consensus.cc:2454] T 6a77bda800d44c4bbe1b082a5ea05b55 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c15e23a34059456788f9775031e27ec0 in term 1.
I20240502 07:59:44.876536 20671 raft_consensus.cc:2454] T 6a77bda800d44c4bbe1b082a5ea05b55 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c15e23a34059456788f9775031e27ec0 in term 1.
I20240502 07:59:44.876552 20275 raft_consensus.cc:2454] T 6a77bda800d44c4bbe1b082a5ea05b55 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c15e23a34059456788f9775031e27ec0 in term 1.
I20240502 07:59:44.877099 20405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a77bda800d44c4bbe1b082a5ea05b55" candidate_uuid: "c15e23a34059456788f9775031e27ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35642aa2099c40e59bc2092604078e07"
I20240502 07:59:44.877197 20405 raft_consensus.cc:3037] T 6a77bda800d44c4bbe1b082a5ea05b55 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:44.877194 20143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a77bda800d44c4bbe1b082a5ea05b55" candidate_uuid: "c15e23a34059456788f9775031e27ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7180fe4574c09a8dce7c27500aab2"
I20240502 07:59:44.877288 20143 raft_consensus.cc:3037] T 6a77bda800d44c4bbe1b082a5ea05b55 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:44.877529 20738 leader_election.cc:302] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: b443e68b6212450286869124684bdfff, b5f471f8b5944986a7d878cf6ac550b8, b7604c0eb3bf4a7cb260b564a94a1191, c15e23a34059456788f9775031e27ec0; no voters:
I20240502 07:59:44.877671 22054 raft_consensus.cc:2781] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:44.877736 22054 raft_consensus.cc:686] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [term 1 LEADER]: Becoming Leader. State: Replica: c15e23a34059456788f9775031e27ec0, State: Running, Role: LEADER
I20240502 07:59:44.877862 22054 consensus_queue.cc:237] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.878062 20405 raft_consensus.cc:2454] T 6a77bda800d44c4bbe1b082a5ea05b55 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c15e23a34059456788f9775031e27ec0 in term 1.
I20240502 07:59:44.878263 20143 raft_consensus.cc:2454] T 6a77bda800d44c4bbe1b082a5ea05b55 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c15e23a34059456788f9775031e27ec0 in term 1.
I20240502 07:59:44.878306 20539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a77bda800d44c4bbe1b082a5ea05b55" candidate_uuid: "c15e23a34059456788f9775031e27ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e951f009258e473cb57cf8773c7975d4"
I20240502 07:59:44.878407 20539 raft_consensus.cc:3037] T 6a77bda800d44c4bbe1b082a5ea05b55 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:44.879287 20539 raft_consensus.cc:2454] T 6a77bda800d44c4bbe1b082a5ea05b55 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c15e23a34059456788f9775031e27ec0 in term 1.
I20240502 07:59:44.880232 19993 catalog_manager.cc:5484] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 reported cstate change: term changed from 0 to 1, leader changed from <none> to c15e23a34059456788f9775031e27ec0 (127.3.139.70). New cstate: current_term: 1 leader_uuid: "c15e23a34059456788f9775031e27ec0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:44.948998 22061 raft_consensus.cc:482] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:44.949120 22061 raft_consensus.cc:504] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.949755 22061 leader_election.cc:288] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b443e68b6212450286869124684bdfff (127.3.139.69:37923), e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315), c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085), b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577), 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703), 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587)
I20240502 07:59:44.950008 20671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae6d597fbc9242539edee8685a015d99" candidate_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b443e68b6212450286869124684bdfff" is_pre_election: true
I20240502 07:59:44.950006 20802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae6d597fbc9242539edee8685a015d99" candidate_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15e23a34059456788f9775031e27ec0" is_pre_election: true
I20240502 07:59:44.950088 20405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae6d597fbc9242539edee8685a015d99" candidate_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35642aa2099c40e59bc2092604078e07" is_pre_election: true
I20240502 07:59:44.950127 20671 raft_consensus.cc:2454] T ae6d597fbc9242539edee8685a015d99 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7604c0eb3bf4a7cb260b564a94a1191 in term 0.
I20240502 07:59:44.950155 20802 raft_consensus.cc:2454] T ae6d597fbc9242539edee8685a015d99 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7604c0eb3bf4a7cb260b564a94a1191 in term 0.
I20240502 07:59:44.950191 20405 raft_consensus.cc:2454] T ae6d597fbc9242539edee8685a015d99 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7604c0eb3bf4a7cb260b564a94a1191 in term 0.
I20240502 07:59:44.950592 20934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae6d597fbc9242539edee8685a015d99" candidate_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f471f8b5944986a7d878cf6ac550b8" is_pre_election: true
I20240502 07:59:44.950625 20209 leader_election.cc:302] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 35642aa2099c40e59bc2092604078e07, b443e68b6212450286869124684bdfff, b7604c0eb3bf4a7cb260b564a94a1191, c15e23a34059456788f9775031e27ec0; no voters:
I20240502 07:59:44.950680 20143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae6d597fbc9242539edee8685a015d99" candidate_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7180fe4574c09a8dce7c27500aab2" is_pre_election: true
I20240502 07:59:44.950718 20934 raft_consensus.cc:2454] T ae6d597fbc9242539edee8685a015d99 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7604c0eb3bf4a7cb260b564a94a1191 in term 0.
I20240502 07:59:44.950786 22061 raft_consensus.cc:2781] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:44.950796 20143 raft_consensus.cc:2454] T ae6d597fbc9242539edee8685a015d99 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7604c0eb3bf4a7cb260b564a94a1191 in term 0.
I20240502 07:59:44.950839 22061 raft_consensus.cc:482] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:44.950878 22061 raft_consensus.cc:3037] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:44.950990 20539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae6d597fbc9242539edee8685a015d99" candidate_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e951f009258e473cb57cf8773c7975d4" is_pre_election: true
I20240502 07:59:44.951124 20539 raft_consensus.cc:2454] T ae6d597fbc9242539edee8685a015d99 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7604c0eb3bf4a7cb260b564a94a1191 in term 0.
I20240502 07:59:44.951743 22061 raft_consensus.cc:504] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.952188 22061 leader_election.cc:288] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [CANDIDATE]: Term 1 election: Requested vote from peers b443e68b6212450286869124684bdfff (127.3.139.69:37923), e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315), c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085), b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577), 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703), 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587)
I20240502 07:59:44.952445 20802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae6d597fbc9242539edee8685a015d99" candidate_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15e23a34059456788f9775031e27ec0"
I20240502 07:59:44.952471 20405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae6d597fbc9242539edee8685a015d99" candidate_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35642aa2099c40e59bc2092604078e07"
I20240502 07:59:44.952512 20934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae6d597fbc9242539edee8685a015d99" candidate_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f471f8b5944986a7d878cf6ac550b8"
I20240502 07:59:44.952551 20802 raft_consensus.cc:3037] T ae6d597fbc9242539edee8685a015d99 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:44.952567 20405 raft_consensus.cc:3037] T ae6d597fbc9242539edee8685a015d99 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:44.952598 20934 raft_consensus.cc:3037] T ae6d597fbc9242539edee8685a015d99 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:44.952605 20143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae6d597fbc9242539edee8685a015d99" candidate_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7180fe4574c09a8dce7c27500aab2"
I20240502 07:59:44.952701 20143 raft_consensus.cc:3037] T ae6d597fbc9242539edee8685a015d99 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:44.953403 20934 raft_consensus.cc:2454] T ae6d597fbc9242539edee8685a015d99 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7604c0eb3bf4a7cb260b564a94a1191 in term 1.
I20240502 07:59:44.953404 20802 raft_consensus.cc:2454] T ae6d597fbc9242539edee8685a015d99 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7604c0eb3bf4a7cb260b564a94a1191 in term 1.
I20240502 07:59:44.953469 20405 raft_consensus.cc:2454] T ae6d597fbc9242539edee8685a015d99 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7604c0eb3bf4a7cb260b564a94a1191 in term 1.
I20240502 07:59:44.953512 20143 raft_consensus.cc:2454] T ae6d597fbc9242539edee8685a015d99 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7604c0eb3bf4a7cb260b564a94a1191 in term 1.
I20240502 07:59:44.953719 20211 leader_election.cc:302] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 35642aa2099c40e59bc2092604078e07, b5f471f8b5944986a7d878cf6ac550b8, b7604c0eb3bf4a7cb260b564a94a1191, c15e23a34059456788f9775031e27ec0; no voters:
I20240502 07:59:44.953796 20539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae6d597fbc9242539edee8685a015d99" candidate_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e951f009258e473cb57cf8773c7975d4"
I20240502 07:59:44.953869 22061 raft_consensus.cc:2781] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:44.953900 20539 raft_consensus.cc:3037] T ae6d597fbc9242539edee8685a015d99 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:44.953930 22061 raft_consensus.cc:686] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 LEADER]: Becoming Leader. State: Replica: b7604c0eb3bf4a7cb260b564a94a1191, State: Running, Role: LEADER
I20240502 07:59:44.953939 20671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae6d597fbc9242539edee8685a015d99" candidate_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b443e68b6212450286869124684bdfff"
I20240502 07:59:44.954021 20671 raft_consensus.cc:3037] T ae6d597fbc9242539edee8685a015d99 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:44.954089 22061 consensus_queue.cc:237] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:44.954990 20539 raft_consensus.cc:2454] T ae6d597fbc9242539edee8685a015d99 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7604c0eb3bf4a7cb260b564a94a1191 in term 1.
I20240502 07:59:44.955761 20671 raft_consensus.cc:2454] T ae6d597fbc9242539edee8685a015d99 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7604c0eb3bf4a7cb260b564a94a1191 in term 1.
I20240502 07:59:44.955965 19993 catalog_manager.cc:5484] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 reported cstate change: term changed from 0 to 1, leader changed from <none> to b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66). New cstate: current_term: 1 leader_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:45.077348 22099 raft_consensus.cc:482] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:45.077481 22099 raft_consensus.cc:504] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.078066 22099 leader_election.cc:288] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315), c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085), b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577), b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331), 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703), 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587)
I20240502 07:59:45.078253 20537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f03a9b07a8c40bdbdf3ff714302a3ea" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e951f009258e473cb57cf8773c7975d4" is_pre_election: true
I20240502 07:59:45.078320 20802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f03a9b07a8c40bdbdf3ff714302a3ea" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15e23a34059456788f9775031e27ec0" is_pre_election: true
I20240502 07:59:45.078397 20537 raft_consensus.cc:2454] T 0f03a9b07a8c40bdbdf3ff714302a3ea P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b443e68b6212450286869124684bdfff in term 0.
I20240502 07:59:45.078424 20802 raft_consensus.cc:2454] T 0f03a9b07a8c40bdbdf3ff714302a3ea P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b443e68b6212450286869124684bdfff in term 0.
I20240502 07:59:45.078389 20143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f03a9b07a8c40bdbdf3ff714302a3ea" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7180fe4574c09a8dce7c27500aab2" is_pre_election: true
I20240502 07:59:45.078514 20143 raft_consensus.cc:2454] T 0f03a9b07a8c40bdbdf3ff714302a3ea P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b443e68b6212450286869124684bdfff in term 0.
I20240502 07:59:45.078522 20405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f03a9b07a8c40bdbdf3ff714302a3ea" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35642aa2099c40e59bc2092604078e07" is_pre_election: true
I20240502 07:59:45.078631 20405 raft_consensus.cc:2454] T 0f03a9b07a8c40bdbdf3ff714302a3ea P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b443e68b6212450286869124684bdfff in term 0.
I20240502 07:59:45.078747 20605 leader_election.cc:302] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 29a7180fe4574c09a8dce7c27500aab2, b443e68b6212450286869124684bdfff, c15e23a34059456788f9775031e27ec0, e951f009258e473cb57cf8773c7975d4; no voters:
I20240502 07:59:45.078913 22099 raft_consensus.cc:2781] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:45.078886 20275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f03a9b07a8c40bdbdf3ff714302a3ea" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" is_pre_election: true
I20240502 07:59:45.078966 22099 raft_consensus.cc:482] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:45.079007 22099 raft_consensus.cc:3037] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:45.078998 20275 raft_consensus.cc:2454] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b443e68b6212450286869124684bdfff in term 0.
I20240502 07:59:45.079068 20934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f03a9b07a8c40bdbdf3ff714302a3ea" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f471f8b5944986a7d878cf6ac550b8" is_pre_election: true
I20240502 07:59:45.079200 20934 raft_consensus.cc:2454] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b443e68b6212450286869124684bdfff in term 0.
I20240502 07:59:45.079964 22099 raft_consensus.cc:504] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.080426 22099 leader_election.cc:288] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [CANDIDATE]: Term 1 election: Requested vote from peers e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315), c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085), b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577), b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331), 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703), 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587)
I20240502 07:59:45.080608 20537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f03a9b07a8c40bdbdf3ff714302a3ea" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e951f009258e473cb57cf8773c7975d4"
I20240502 07:59:45.080713 20537 raft_consensus.cc:3037] T 0f03a9b07a8c40bdbdf3ff714302a3ea P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:45.080773 20802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f03a9b07a8c40bdbdf3ff714302a3ea" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15e23a34059456788f9775031e27ec0"
I20240502 07:59:45.080791 20275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f03a9b07a8c40bdbdf3ff714302a3ea" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7604c0eb3bf4a7cb260b564a94a1191"
I20240502 07:59:45.080869 20275 raft_consensus.cc:3037] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:45.080869 20802 raft_consensus.cc:3037] T 0f03a9b07a8c40bdbdf3ff714302a3ea P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:45.080914 20405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f03a9b07a8c40bdbdf3ff714302a3ea" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35642aa2099c40e59bc2092604078e07"
I20240502 07:59:45.081007 20405 raft_consensus.cc:3037] T 0f03a9b07a8c40bdbdf3ff714302a3ea P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:45.081640 20537 raft_consensus.cc:2454] T 0f03a9b07a8c40bdbdf3ff714302a3ea P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b443e68b6212450286869124684bdfff in term 1.
I20240502 07:59:45.081753 20802 raft_consensus.cc:2454] T 0f03a9b07a8c40bdbdf3ff714302a3ea P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b443e68b6212450286869124684bdfff in term 1.
I20240502 07:59:45.081754 20275 raft_consensus.cc:2454] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b443e68b6212450286869124684bdfff in term 1.
I20240502 07:59:45.081871 20405 raft_consensus.cc:2454] T 0f03a9b07a8c40bdbdf3ff714302a3ea P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b443e68b6212450286869124684bdfff in term 1.
I20240502 07:59:45.082051 20606 leader_election.cc:302] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: b443e68b6212450286869124684bdfff, b7604c0eb3bf4a7cb260b564a94a1191, c15e23a34059456788f9775031e27ec0, e951f009258e473cb57cf8773c7975d4; no voters:
I20240502 07:59:45.082085 20934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f03a9b07a8c40bdbdf3ff714302a3ea" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f471f8b5944986a7d878cf6ac550b8"
I20240502 07:59:45.082195 20934 raft_consensus.cc:3037] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:45.082310 22099 raft_consensus.cc:2781] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:45.082278 20143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f03a9b07a8c40bdbdf3ff714302a3ea" candidate_uuid: "b443e68b6212450286869124684bdfff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7180fe4574c09a8dce7c27500aab2"
I20240502 07:59:45.082379 22099 raft_consensus.cc:686] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [term 1 LEADER]: Becoming Leader. State: Replica: b443e68b6212450286869124684bdfff, State: Running, Role: LEADER
I20240502 07:59:45.082384 20143 raft_consensus.cc:3037] T 0f03a9b07a8c40bdbdf3ff714302a3ea P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:45.082515 22099 consensus_queue.cc:237] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.083161 20934 raft_consensus.cc:2454] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b443e68b6212450286869124684bdfff in term 1.
I20240502 07:59:45.083549 20143 raft_consensus.cc:2454] T 0f03a9b07a8c40bdbdf3ff714302a3ea P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b443e68b6212450286869124684bdfff in term 1.
I20240502 07:59:45.085249 19993 catalog_manager.cc:5484] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff reported cstate change: term changed from 0 to 1, leader changed from <none> to b443e68b6212450286869124684bdfff (127.3.139.69). New cstate: current_term: 1 leader_uuid: "b443e68b6212450286869124684bdfff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: UNKNOWN } } }
W20240502 07:59:45.216202 22151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:45.216446 22151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:45.220584 22151 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_5'
I20240502 07:59:45.300681 20143 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:45.300698 20935 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:45.301122 20802 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:45.301203 20671 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:45.301904 20405 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:45.302425 22062 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:45.302541 20405 raft_consensus.cc:1261] T 0f03a9b07a8c40bdbdf3ff714302a3ea P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:45.302714 22062 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:45.302883 22099 consensus_queue.cc:1035] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:45.303182 20143 raft_consensus.cc:1261] T 0f03a9b07a8c40bdbdf3ff714302a3ea P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:45.303308 20140 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:45.303529 20935 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:45.303650 20275 raft_consensus.cc:1261] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:45.303942 20537 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:45.304201 20539 raft_consensus.cc:1261] T 0f03a9b07a8c40bdbdf3ff714302a3ea P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:45.304483 21918 consensus_queue.cc:1035] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:45.306188 22183 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:45.306488 22061 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:45.306757 22061 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:45.306748 21801 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:45.306994 21801 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:45.307245 21801 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:45.307735 22187 consensus_queue.cc:1035] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:45.307868 22099 consensus_queue.cc:1035] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:45.309085 20273 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:45.310566 20671 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:45.314386 22186 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:45.314793 20802 raft_consensus.cc:1261] T 0f03a9b07a8c40bdbdf3ff714302a3ea P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:45.315181 20537 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:45.315199 22099 consensus_queue.cc:1035] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:45.322489 22186 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:45.328599 22061 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:45.330466 20405 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:45.332764 20935 raft_consensus.cc:1261] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:45.333101 22054 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:45.334156 21918 consensus_queue.cc:1035] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 211.59 ms
time per row: 0.105795 ms
I20240502 07:59:45.473691 19992 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33518:
name: "rebalance_test_table_6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 7
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240502 07:59:45.474187 19992 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_6 in case of a server failure: 8 tablet servers would be needed, 7 are available. Consider bringing up more tablet servers.
I20240502 07:59:45.479348 20517 tablet_service.cc:1467] Processing CreateTablet for tablet 574fdd7a3c114ac9928ac12d1a7a4186 (DEFAULT_TABLE table=rebalance_test_table_6 [id=db1e20e82b894565a302e53bd7791607]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:45.479424 20782 tablet_service.cc:1467] Processing CreateTablet for tablet 574fdd7a3c114ac9928ac12d1a7a4186 (DEFAULT_TABLE table=rebalance_test_table_6 [id=db1e20e82b894565a302e53bd7791607]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:45.479660 20517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 574fdd7a3c114ac9928ac12d1a7a4186. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:45.479729 20782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 574fdd7a3c114ac9928ac12d1a7a4186. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:45.481333 22254 tablet_bootstrap.cc:492] T 574fdd7a3c114ac9928ac12d1a7a4186 P e951f009258e473cb57cf8773c7975d4: Bootstrap starting.
I20240502 07:59:45.482054 20121 tablet_service.cc:1467] Processing CreateTablet for tablet 574fdd7a3c114ac9928ac12d1a7a4186 (DEFAULT_TABLE table=rebalance_test_table_6 [id=db1e20e82b894565a302e53bd7791607]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:45.482107 20913 tablet_service.cc:1467] Processing CreateTablet for tablet fb0f04b72dc046c0afee40b895e17a1d (DEFAULT_TABLE table=rebalance_test_table_6 [id=db1e20e82b894565a302e53bd7791607]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:45.482331 20121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 574fdd7a3c114ac9928ac12d1a7a4186. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:45.482388 20913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb0f04b72dc046c0afee40b895e17a1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:45.482450 22254 tablet_bootstrap.cc:654] T 574fdd7a3c114ac9928ac12d1a7a4186 P e951f009258e473cb57cf8773c7975d4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:45.482421 20915 tablet_service.cc:1467] Processing CreateTablet for tablet 9ad40762dc784dfa9f3507e025db32b8 (DEFAULT_TABLE table=rebalance_test_table_6 [id=db1e20e82b894565a302e53bd7791607]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:45.482630 20915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ad40762dc784dfa9f3507e025db32b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:45.483568 22254 tablet_bootstrap.cc:492] T 574fdd7a3c114ac9928ac12d1a7a4186 P e951f009258e473cb57cf8773c7975d4: No bootstrap required, opened a new log
I20240502 07:59:45.483659 22254 ts_tablet_manager.cc:1367] T 574fdd7a3c114ac9928ac12d1a7a4186 P e951f009258e473cb57cf8773c7975d4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:45.483637 20651 tablet_service.cc:1467] Processing CreateTablet for tablet 574fdd7a3c114ac9928ac12d1a7a4186 (DEFAULT_TABLE table=rebalance_test_table_6 [id=db1e20e82b894565a302e53bd7791607]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:45.483891 20651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 574fdd7a3c114ac9928ac12d1a7a4186. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:45.484050 22254 raft_consensus.cc:348] T 574fdd7a3c114ac9928ac12d1a7a4186 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.484162 22254 raft_consensus.cc:374] T 574fdd7a3c114ac9928ac12d1a7a4186 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:45.484192 22254 raft_consensus.cc:729] T 574fdd7a3c114ac9928ac12d1a7a4186 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e951f009258e473cb57cf8773c7975d4, State: Initialized, Role: FOLLOWER
I20240502 07:59:45.484304 22254 consensus_queue.cc:260] T 574fdd7a3c114ac9928ac12d1a7a4186 P e951f009258e473cb57cf8773c7975d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.484586 22254 ts_tablet_manager.cc:1398] T 574fdd7a3c114ac9928ac12d1a7a4186 P e951f009258e473cb57cf8773c7975d4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:45.484946 20650 tablet_service.cc:1467] Processing CreateTablet for tablet fb0f04b72dc046c0afee40b895e17a1d (DEFAULT_TABLE table=rebalance_test_table_6 [id=db1e20e82b894565a302e53bd7791607]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:45.485158 20650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb0f04b72dc046c0afee40b895e17a1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:45.485719 22259 tablet_bootstrap.cc:492] T 574fdd7a3c114ac9928ac12d1a7a4186 P b443e68b6212450286869124684bdfff: Bootstrap starting.
I20240502 07:59:45.485891 20649 tablet_service.cc:1467] Processing CreateTablet for tablet 9ad40762dc784dfa9f3507e025db32b8 (DEFAULT_TABLE table=rebalance_test_table_6 [id=db1e20e82b894565a302e53bd7791607]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:45.486078 20649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ad40762dc784dfa9f3507e025db32b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:45.486186 22258 tablet_bootstrap.cc:492] T fb0f04b72dc046c0afee40b895e17a1d P b5f471f8b5944986a7d878cf6ac550b8: Bootstrap starting.
I20240502 07:59:45.486553 22259 tablet_bootstrap.cc:654] T 574fdd7a3c114ac9928ac12d1a7a4186 P b443e68b6212450286869124684bdfff: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:45.487099 22258 tablet_bootstrap.cc:654] T fb0f04b72dc046c0afee40b895e17a1d P b5f471f8b5944986a7d878cf6ac550b8: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:45.487687 20253 tablet_service.cc:1467] Processing CreateTablet for tablet 574fdd7a3c114ac9928ac12d1a7a4186 (DEFAULT_TABLE table=rebalance_test_table_6 [id=db1e20e82b894565a302e53bd7791607]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:45.487756 22257 tablet_bootstrap.cc:492] T 574fdd7a3c114ac9928ac12d1a7a4186 P 29a7180fe4574c09a8dce7c27500aab2: Bootstrap starting.
I20240502 07:59:45.487874 20255 tablet_service.cc:1467] Processing CreateTablet for tablet 9ad40762dc784dfa9f3507e025db32b8 (DEFAULT_TABLE table=rebalance_test_table_6 [id=db1e20e82b894565a302e53bd7791607]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:45.487941 20253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 574fdd7a3c114ac9928ac12d1a7a4186. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:45.488061 20255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ad40762dc784dfa9f3507e025db32b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:45.488610 22257 tablet_bootstrap.cc:654] T 574fdd7a3c114ac9928ac12d1a7a4186 P 29a7180fe4574c09a8dce7c27500aab2: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:45.489495 22255 tablet_bootstrap.cc:492] T 574fdd7a3c114ac9928ac12d1a7a4186 P c15e23a34059456788f9775031e27ec0: Bootstrap starting.
I20240502 07:59:45.489881 22257 tablet_bootstrap.cc:492] T 574fdd7a3c114ac9928ac12d1a7a4186 P 29a7180fe4574c09a8dce7c27500aab2: No bootstrap required, opened a new log
I20240502 07:59:45.489970 22257 ts_tablet_manager.cc:1367] T 574fdd7a3c114ac9928ac12d1a7a4186 P 29a7180fe4574c09a8dce7c27500aab2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:45.487694 20254 tablet_service.cc:1467] Processing CreateTablet for tablet fb0f04b72dc046c0afee40b895e17a1d (DEFAULT_TABLE table=rebalance_test_table_6 [id=db1e20e82b894565a302e53bd7791607]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:45.490059 22263 tablet_bootstrap.cc:492] T 9ad40762dc784dfa9f3507e025db32b8 P b7604c0eb3bf4a7cb260b564a94a1191: Bootstrap starting.
I20240502 07:59:45.482106 20914 tablet_service.cc:1467] Processing CreateTablet for tablet 574fdd7a3c114ac9928ac12d1a7a4186 (DEFAULT_TABLE table=rebalance_test_table_6 [id=db1e20e82b894565a302e53bd7791607]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:45.490247 20254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb0f04b72dc046c0afee40b895e17a1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:45.490283 22255 tablet_bootstrap.cc:654] T 574fdd7a3c114ac9928ac12d1a7a4186 P c15e23a34059456788f9775031e27ec0: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:45.490324 20914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 574fdd7a3c114ac9928ac12d1a7a4186. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:45.490563 22257 raft_consensus.cc:348] T 574fdd7a3c114ac9928ac12d1a7a4186 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.490689 22257 raft_consensus.cc:374] T 574fdd7a3c114ac9928ac12d1a7a4186 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:45.490720 22257 raft_consensus.cc:729] T 574fdd7a3c114ac9928ac12d1a7a4186 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29a7180fe4574c09a8dce7c27500aab2, State: Initialized, Role: FOLLOWER
I20240502 07:59:45.490839 22257 consensus_queue.cc:260] T 574fdd7a3c114ac9928ac12d1a7a4186 P 29a7180fe4574c09a8dce7c27500aab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.491144 22257 ts_tablet_manager.cc:1398] T 574fdd7a3c114ac9928ac12d1a7a4186 P 29a7180fe4574c09a8dce7c27500aab2: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 07:59:45.493184 22263 tablet_bootstrap.cc:654] T 9ad40762dc784dfa9f3507e025db32b8 P b7604c0eb3bf4a7cb260b564a94a1191: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:45.493115 20387 tablet_service.cc:1467] Processing CreateTablet for tablet 574fdd7a3c114ac9928ac12d1a7a4186 (DEFAULT_TABLE table=rebalance_test_table_6 [id=db1e20e82b894565a302e53bd7791607]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:45.493384 20387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 574fdd7a3c114ac9928ac12d1a7a4186. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:45.493477 22258 tablet_bootstrap.cc:492] T fb0f04b72dc046c0afee40b895e17a1d P b5f471f8b5944986a7d878cf6ac550b8: No bootstrap required, opened a new log
I20240502 07:59:45.493573 22258 ts_tablet_manager.cc:1367] T fb0f04b72dc046c0afee40b895e17a1d P b5f471f8b5944986a7d878cf6ac550b8: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240502 07:59:45.493949 22258 raft_consensus.cc:348] T fb0f04b72dc046c0afee40b895e17a1d P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.494135 22258 raft_consensus.cc:374] T fb0f04b72dc046c0afee40b895e17a1d P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:45.494177 22258 raft_consensus.cc:729] T fb0f04b72dc046c0afee40b895e17a1d P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5f471f8b5944986a7d878cf6ac550b8, State: Initialized, Role: FOLLOWER
I20240502 07:59:45.494297 22258 consensus_queue.cc:260] T fb0f04b72dc046c0afee40b895e17a1d P b5f471f8b5944986a7d878cf6ac550b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.494431 22263 tablet_bootstrap.cc:492] T 9ad40762dc784dfa9f3507e025db32b8 P b7604c0eb3bf4a7cb260b564a94a1191: No bootstrap required, opened a new log
I20240502 07:59:45.494524 22263 ts_tablet_manager.cc:1367] T 9ad40762dc784dfa9f3507e025db32b8 P b7604c0eb3bf4a7cb260b564a94a1191: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 07:59:45.494547 22258 ts_tablet_manager.cc:1398] T fb0f04b72dc046c0afee40b895e17a1d P b5f471f8b5944986a7d878cf6ac550b8: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:45.494709 22258 tablet_bootstrap.cc:492] T 574fdd7a3c114ac9928ac12d1a7a4186 P b5f471f8b5944986a7d878cf6ac550b8: Bootstrap starting.
I20240502 07:59:45.494917 22263 raft_consensus.cc:348] T 9ad40762dc784dfa9f3507e025db32b8 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.495069 22263 raft_consensus.cc:374] T 9ad40762dc784dfa9f3507e025db32b8 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:45.495123 22263 raft_consensus.cc:729] T 9ad40762dc784dfa9f3507e025db32b8 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7604c0eb3bf4a7cb260b564a94a1191, State: Initialized, Role: FOLLOWER
I20240502 07:59:45.495247 22263 consensus_queue.cc:260] T 9ad40762dc784dfa9f3507e025db32b8 P b7604c0eb3bf4a7cb260b564a94a1191 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.495275 20386 tablet_service.cc:1467] Processing CreateTablet for tablet 9ad40762dc784dfa9f3507e025db32b8 (DEFAULT_TABLE table=rebalance_test_table_6 [id=db1e20e82b894565a302e53bd7791607]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:45.495477 20386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ad40762dc784dfa9f3507e025db32b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:45.495590 22263 ts_tablet_manager.cc:1398] T 9ad40762dc784dfa9f3507e025db32b8 P b7604c0eb3bf4a7cb260b564a94a1191: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:45.495745 22263 tablet_bootstrap.cc:492] T fb0f04b72dc046c0afee40b895e17a1d P b7604c0eb3bf4a7cb260b564a94a1191: Bootstrap starting.
I20240502 07:59:45.496016 22258 tablet_bootstrap.cc:654] T 574fdd7a3c114ac9928ac12d1a7a4186 P b5f471f8b5944986a7d878cf6ac550b8: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:45.496497 22263 tablet_bootstrap.cc:654] T fb0f04b72dc046c0afee40b895e17a1d P b7604c0eb3bf4a7cb260b564a94a1191: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:45.497061 22258 tablet_bootstrap.cc:492] T 574fdd7a3c114ac9928ac12d1a7a4186 P b5f471f8b5944986a7d878cf6ac550b8: No bootstrap required, opened a new log
I20240502 07:59:45.493115 20385 tablet_service.cc:1467] Processing CreateTablet for tablet fb0f04b72dc046c0afee40b895e17a1d (DEFAULT_TABLE table=rebalance_test_table_6 [id=db1e20e82b894565a302e53bd7791607]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:45.497203 20385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb0f04b72dc046c0afee40b895e17a1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:45.497205 22258 ts_tablet_manager.cc:1367] T 574fdd7a3c114ac9928ac12d1a7a4186 P b5f471f8b5944986a7d878cf6ac550b8: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 07:59:45.497736 22258 raft_consensus.cc:348] T 574fdd7a3c114ac9928ac12d1a7a4186 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.497831 22258 raft_consensus.cc:374] T 574fdd7a3c114ac9928ac12d1a7a4186 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:45.497860 22258 raft_consensus.cc:729] T 574fdd7a3c114ac9928ac12d1a7a4186 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5f471f8b5944986a7d878cf6ac550b8, State: Initialized, Role: FOLLOWER
I20240502 07:59:45.497974 22258 consensus_queue.cc:260] T 574fdd7a3c114ac9928ac12d1a7a4186 P b5f471f8b5944986a7d878cf6ac550b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.498355 22258 ts_tablet_manager.cc:1398] T 574fdd7a3c114ac9928ac12d1a7a4186 P b5f471f8b5944986a7d878cf6ac550b8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:45.498836 22255 tablet_bootstrap.cc:492] T 574fdd7a3c114ac9928ac12d1a7a4186 P c15e23a34059456788f9775031e27ec0: No bootstrap required, opened a new log
I20240502 07:59:45.498922 22255 ts_tablet_manager.cc:1367] T 574fdd7a3c114ac9928ac12d1a7a4186 P c15e23a34059456788f9775031e27ec0: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20240502 07:59:45.499357 20519 tablet_service.cc:1467] Processing CreateTablet for tablet 9ad40762dc784dfa9f3507e025db32b8 (DEFAULT_TABLE table=rebalance_test_table_6 [id=db1e20e82b894565a302e53bd7791607]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:45.499375 22255 raft_consensus.cc:348] T 574fdd7a3c114ac9928ac12d1a7a4186 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.499468 22255 raft_consensus.cc:374] T 574fdd7a3c114ac9928ac12d1a7a4186 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:45.499519 22255 raft_consensus.cc:729] T 574fdd7a3c114ac9928ac12d1a7a4186 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c15e23a34059456788f9775031e27ec0, State: Initialized, Role: FOLLOWER
I20240502 07:59:45.499604 20519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ad40762dc784dfa9f3507e025db32b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:45.499716 22255 consensus_queue.cc:260] T 574fdd7a3c114ac9928ac12d1a7a4186 P c15e23a34059456788f9775031e27ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.499989 22255 ts_tablet_manager.cc:1398] T 574fdd7a3c114ac9928ac12d1a7a4186 P c15e23a34059456788f9775031e27ec0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:45.501484 22254 tablet_bootstrap.cc:492] T 9ad40762dc784dfa9f3507e025db32b8 P e951f009258e473cb57cf8773c7975d4: Bootstrap starting.
I20240502 07:59:45.501631 20517 tablet_service.cc:1467] Processing CreateTablet for tablet fb0f04b72dc046c0afee40b895e17a1d (DEFAULT_TABLE table=rebalance_test_table_6 [id=db1e20e82b894565a302e53bd7791607]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:45.501847 20517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb0f04b72dc046c0afee40b895e17a1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:45.502239 22254 tablet_bootstrap.cc:654] T 9ad40762dc784dfa9f3507e025db32b8 P e951f009258e473cb57cf8773c7975d4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:45.503106 22254 tablet_bootstrap.cc:492] T 9ad40762dc784dfa9f3507e025db32b8 P e951f009258e473cb57cf8773c7975d4: No bootstrap required, opened a new log
I20240502 07:59:45.503190 22254 ts_tablet_manager.cc:1367] T 9ad40762dc784dfa9f3507e025db32b8 P e951f009258e473cb57cf8773c7975d4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:45.503520 22254 raft_consensus.cc:348] T 9ad40762dc784dfa9f3507e025db32b8 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.503609 22254 raft_consensus.cc:374] T 9ad40762dc784dfa9f3507e025db32b8 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:45.503636 22254 raft_consensus.cc:729] T 9ad40762dc784dfa9f3507e025db32b8 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e951f009258e473cb57cf8773c7975d4, State: Initialized, Role: FOLLOWER
I20240502 07:59:45.503722 22254 consensus_queue.cc:260] T 9ad40762dc784dfa9f3507e025db32b8 P e951f009258e473cb57cf8773c7975d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.503868 20121 tablet_service.cc:1467] Processing CreateTablet for tablet fb0f04b72dc046c0afee40b895e17a1d (DEFAULT_TABLE table=rebalance_test_table_6 [id=db1e20e82b894565a302e53bd7791607]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:45.503988 22254 ts_tablet_manager.cc:1398] T 9ad40762dc784dfa9f3507e025db32b8 P e951f009258e473cb57cf8773c7975d4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:45.504142 22254 tablet_bootstrap.cc:492] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4: Bootstrap starting.
I20240502 07:59:45.504083 20122 tablet_service.cc:1467] Processing CreateTablet for tablet 9ad40762dc784dfa9f3507e025db32b8 (DEFAULT_TABLE table=rebalance_test_table_6 [id=db1e20e82b894565a302e53bd7791607]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:45.504334 20781 tablet_service.cc:1467] Processing CreateTablet for tablet 9ad40762dc784dfa9f3507e025db32b8 (DEFAULT_TABLE table=rebalance_test_table_6 [id=db1e20e82b894565a302e53bd7791607]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:45.504586 20781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ad40762dc784dfa9f3507e025db32b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:45.504555 20782 tablet_service.cc:1467] Processing CreateTablet for tablet fb0f04b72dc046c0afee40b895e17a1d (DEFAULT_TABLE table=rebalance_test_table_6 [id=db1e20e82b894565a302e53bd7791607]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:45.504776 20782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb0f04b72dc046c0afee40b895e17a1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:45.504094 20121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb0f04b72dc046c0afee40b895e17a1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:45.505059 22254 tablet_bootstrap.cc:654] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:45.506531 22255 tablet_bootstrap.cc:492] T fb0f04b72dc046c0afee40b895e17a1d P c15e23a34059456788f9775031e27ec0: Bootstrap starting.
I20240502 07:59:45.506912 22257 tablet_bootstrap.cc:492] T fb0f04b72dc046c0afee40b895e17a1d P 29a7180fe4574c09a8dce7c27500aab2: Bootstrap starting.
I20240502 07:59:45.507532 22255 tablet_bootstrap.cc:654] T fb0f04b72dc046c0afee40b895e17a1d P c15e23a34059456788f9775031e27ec0: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:45.507769 22257 tablet_bootstrap.cc:654] T fb0f04b72dc046c0afee40b895e17a1d P 29a7180fe4574c09a8dce7c27500aab2: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:45.507931 20122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ad40762dc784dfa9f3507e025db32b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:45.511523 22257 tablet_bootstrap.cc:492] T fb0f04b72dc046c0afee40b895e17a1d P 29a7180fe4574c09a8dce7c27500aab2: No bootstrap required, opened a new log
I20240502 07:59:45.511602 22257 ts_tablet_manager.cc:1367] T fb0f04b72dc046c0afee40b895e17a1d P 29a7180fe4574c09a8dce7c27500aab2: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:59:45.512001 22257 raft_consensus.cc:348] T fb0f04b72dc046c0afee40b895e17a1d P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.512109 22257 raft_consensus.cc:374] T fb0f04b72dc046c0afee40b895e17a1d P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:45.512140 22257 raft_consensus.cc:729] T fb0f04b72dc046c0afee40b895e17a1d P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29a7180fe4574c09a8dce7c27500aab2, State: Initialized, Role: FOLLOWER
I20240502 07:59:45.512240 22257 consensus_queue.cc:260] T fb0f04b72dc046c0afee40b895e17a1d P 29a7180fe4574c09a8dce7c27500aab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.512483 22257 ts_tablet_manager.cc:1398] T fb0f04b72dc046c0afee40b895e17a1d P 29a7180fe4574c09a8dce7c27500aab2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:45.512638 22257 tablet_bootstrap.cc:492] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2: Bootstrap starting.
I20240502 07:59:45.513391 22257 tablet_bootstrap.cc:654] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:45.513901 22272 tablet_bootstrap.cc:492] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07: Bootstrap starting.
I20240502 07:59:45.514355 22263 tablet_bootstrap.cc:492] T fb0f04b72dc046c0afee40b895e17a1d P b7604c0eb3bf4a7cb260b564a94a1191: No bootstrap required, opened a new log
I20240502 07:59:45.514436 22263 ts_tablet_manager.cc:1367] T fb0f04b72dc046c0afee40b895e17a1d P b7604c0eb3bf4a7cb260b564a94a1191: Time spent bootstrapping tablet: real 0.019s user 0.001s sys 0.000s
I20240502 07:59:45.514757 22258 tablet_bootstrap.cc:492] T 9ad40762dc784dfa9f3507e025db32b8 P b5f471f8b5944986a7d878cf6ac550b8: Bootstrap starting.
I20240502 07:59:45.514850 22263 raft_consensus.cc:348] T fb0f04b72dc046c0afee40b895e17a1d P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.514958 22263 raft_consensus.cc:374] T fb0f04b72dc046c0afee40b895e17a1d P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:45.514989 22263 raft_consensus.cc:729] T fb0f04b72dc046c0afee40b895e17a1d P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7604c0eb3bf4a7cb260b564a94a1191, State: Initialized, Role: FOLLOWER
I20240502 07:59:45.515127 22263 consensus_queue.cc:260] T fb0f04b72dc046c0afee40b895e17a1d P b7604c0eb3bf4a7cb260b564a94a1191 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.515379 22263 ts_tablet_manager.cc:1398] T fb0f04b72dc046c0afee40b895e17a1d P b7604c0eb3bf4a7cb260b564a94a1191: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:45.515502 22272 tablet_bootstrap.cc:654] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:45.515527 22263 tablet_bootstrap.cc:492] T 574fdd7a3c114ac9928ac12d1a7a4186 P b7604c0eb3bf4a7cb260b564a94a1191: Bootstrap starting.
I20240502 07:59:45.515965 22258 tablet_bootstrap.cc:654] T 9ad40762dc784dfa9f3507e025db32b8 P b5f471f8b5944986a7d878cf6ac550b8: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:45.516330 22263 tablet_bootstrap.cc:654] T 574fdd7a3c114ac9928ac12d1a7a4186 P b7604c0eb3bf4a7cb260b564a94a1191: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:45.516947 22258 tablet_bootstrap.cc:492] T 9ad40762dc784dfa9f3507e025db32b8 P b5f471f8b5944986a7d878cf6ac550b8: No bootstrap required, opened a new log
I20240502 07:59:45.517024 22258 ts_tablet_manager.cc:1367] T 9ad40762dc784dfa9f3507e025db32b8 P b5f471f8b5944986a7d878cf6ac550b8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:45.517407 22258 raft_consensus.cc:348] T 9ad40762dc784dfa9f3507e025db32b8 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.517510 22258 raft_consensus.cc:374] T 9ad40762dc784dfa9f3507e025db32b8 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:45.517509 22259 tablet_bootstrap.cc:492] T 574fdd7a3c114ac9928ac12d1a7a4186 P b443e68b6212450286869124684bdfff: No bootstrap required, opened a new log
I20240502 07:59:45.517539 22258 raft_consensus.cc:729] T 9ad40762dc784dfa9f3507e025db32b8 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5f471f8b5944986a7d878cf6ac550b8, State: Initialized, Role: FOLLOWER
I20240502 07:59:45.517596 22259 ts_tablet_manager.cc:1367] T 574fdd7a3c114ac9928ac12d1a7a4186 P b443e68b6212450286869124684bdfff: Time spent bootstrapping tablet: real 0.032s user 0.001s sys 0.000s
I20240502 07:59:45.517648 22258 consensus_queue.cc:260] T 9ad40762dc784dfa9f3507e025db32b8 P b5f471f8b5944986a7d878cf6ac550b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.517900 22258 ts_tablet_manager.cc:1398] T 9ad40762dc784dfa9f3507e025db32b8 P b5f471f8b5944986a7d878cf6ac550b8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:45.518023 22259 raft_consensus.cc:348] T 574fdd7a3c114ac9928ac12d1a7a4186 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.518157 22259 raft_consensus.cc:374] T 574fdd7a3c114ac9928ac12d1a7a4186 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:45.518196 22259 raft_consensus.cc:729] T 574fdd7a3c114ac9928ac12d1a7a4186 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b443e68b6212450286869124684bdfff, State: Initialized, Role: FOLLOWER
I20240502 07:59:45.518307 22259 consensus_queue.cc:260] T 574fdd7a3c114ac9928ac12d1a7a4186 P b443e68b6212450286869124684bdfff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.518573 22259 ts_tablet_manager.cc:1398] T 574fdd7a3c114ac9928ac12d1a7a4186 P b443e68b6212450286869124684bdfff: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:45.518723 22259 tablet_bootstrap.cc:492] T fb0f04b72dc046c0afee40b895e17a1d P b443e68b6212450286869124684bdfff: Bootstrap starting.
I20240502 07:59:45.519609 22259 tablet_bootstrap.cc:654] T fb0f04b72dc046c0afee40b895e17a1d P b443e68b6212450286869124684bdfff: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:45.521337 22257 tablet_bootstrap.cc:492] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2: No bootstrap required, opened a new log
I20240502 07:59:45.521406 22263 tablet_bootstrap.cc:492] T 574fdd7a3c114ac9928ac12d1a7a4186 P b7604c0eb3bf4a7cb260b564a94a1191: No bootstrap required, opened a new log
I20240502 07:59:45.521493 22263 ts_tablet_manager.cc:1367] T 574fdd7a3c114ac9928ac12d1a7a4186 P b7604c0eb3bf4a7cb260b564a94a1191: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20240502 07:59:45.521502 22257 ts_tablet_manager.cc:1367] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 07:59:45.521565 22254 tablet_bootstrap.cc:492] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4: No bootstrap required, opened a new log
I20240502 07:59:45.521639 22254 ts_tablet_manager.cc:1367] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.000s
I20240502 07:59:45.521883 22263 raft_consensus.cc:348] T 574fdd7a3c114ac9928ac12d1a7a4186 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.522004 22263 raft_consensus.cc:374] T 574fdd7a3c114ac9928ac12d1a7a4186 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:45.522042 22263 raft_consensus.cc:729] T 574fdd7a3c114ac9928ac12d1a7a4186 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7604c0eb3bf4a7cb260b564a94a1191, State: Initialized, Role: FOLLOWER
I20240502 07:59:45.522033 22254 raft_consensus.cc:348] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.522161 22254 raft_consensus.cc:374] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:45.522202 22254 raft_consensus.cc:729] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e951f009258e473cb57cf8773c7975d4, State: Initialized, Role: FOLLOWER
I20240502 07:59:45.522171 22263 consensus_queue.cc:260] T 574fdd7a3c114ac9928ac12d1a7a4186 P b7604c0eb3bf4a7cb260b564a94a1191 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.522315 22254 consensus_queue.cc:260] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.522228 22257 raft_consensus.cc:348] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.522537 22263 ts_tablet_manager.cc:1398] T 574fdd7a3c114ac9928ac12d1a7a4186 P b7604c0eb3bf4a7cb260b564a94a1191: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:45.522599 22254 ts_tablet_manager.cc:1398] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:45.522756 22257 raft_consensus.cc:374] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:45.522840 22257 raft_consensus.cc:729] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29a7180fe4574c09a8dce7c27500aab2, State: Initialized, Role: FOLLOWER
I20240502 07:59:45.523013 22257 consensus_queue.cc:260] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.523438 22257 ts_tablet_manager.cc:1398] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:45.523939 22272 tablet_bootstrap.cc:492] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07: No bootstrap required, opened a new log
I20240502 07:59:45.524039 22255 tablet_bootstrap.cc:492] T fb0f04b72dc046c0afee40b895e17a1d P c15e23a34059456788f9775031e27ec0: No bootstrap required, opened a new log
I20240502 07:59:45.524042 22272 ts_tablet_manager.cc:1367] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20240502 07:59:45.524128 22255 ts_tablet_manager.cc:1367] T fb0f04b72dc046c0afee40b895e17a1d P c15e23a34059456788f9775031e27ec0: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20240502 07:59:45.524483 22272 raft_consensus.cc:348] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.524533 22255 raft_consensus.cc:348] T fb0f04b72dc046c0afee40b895e17a1d P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.524621 22272 raft_consensus.cc:374] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:45.524643 22255 raft_consensus.cc:374] T fb0f04b72dc046c0afee40b895e17a1d P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:45.524654 22272 raft_consensus.cc:729] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35642aa2099c40e59bc2092604078e07, State: Initialized, Role: FOLLOWER
I20240502 07:59:45.524680 22255 raft_consensus.cc:729] T fb0f04b72dc046c0afee40b895e17a1d P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c15e23a34059456788f9775031e27ec0, State: Initialized, Role: FOLLOWER
I20240502 07:59:45.524772 22272 consensus_queue.cc:260] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.524796 22255 consensus_queue.cc:260] T fb0f04b72dc046c0afee40b895e17a1d P c15e23a34059456788f9775031e27ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.525040 22255 ts_tablet_manager.cc:1398] T fb0f04b72dc046c0afee40b895e17a1d P c15e23a34059456788f9775031e27ec0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:45.525048 22272 ts_tablet_manager.cc:1398] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:45.525182 22255 tablet_bootstrap.cc:492] T 9ad40762dc784dfa9f3507e025db32b8 P c15e23a34059456788f9775031e27ec0: Bootstrap starting.
I20240502 07:59:45.525202 22272 tablet_bootstrap.cc:492] T fb0f04b72dc046c0afee40b895e17a1d P 35642aa2099c40e59bc2092604078e07: Bootstrap starting.
I20240502 07:59:45.525962 22272 tablet_bootstrap.cc:654] T fb0f04b72dc046c0afee40b895e17a1d P 35642aa2099c40e59bc2092604078e07: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:45.526001 22255 tablet_bootstrap.cc:654] T 9ad40762dc784dfa9f3507e025db32b8 P c15e23a34059456788f9775031e27ec0: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:45.528014 22255 tablet_bootstrap.cc:492] T 9ad40762dc784dfa9f3507e025db32b8 P c15e23a34059456788f9775031e27ec0: No bootstrap required, opened a new log
I20240502 07:59:45.528091 22255 ts_tablet_manager.cc:1367] T 9ad40762dc784dfa9f3507e025db32b8 P c15e23a34059456788f9775031e27ec0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:45.528472 22255 raft_consensus.cc:348] T 9ad40762dc784dfa9f3507e025db32b8 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.528579 22255 raft_consensus.cc:374] T 9ad40762dc784dfa9f3507e025db32b8 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:45.528609 22255 raft_consensus.cc:729] T 9ad40762dc784dfa9f3507e025db32b8 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c15e23a34059456788f9775031e27ec0, State: Initialized, Role: FOLLOWER
I20240502 07:59:45.528719 22255 consensus_queue.cc:260] T 9ad40762dc784dfa9f3507e025db32b8 P c15e23a34059456788f9775031e27ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.528982 22255 ts_tablet_manager.cc:1398] T 9ad40762dc784dfa9f3507e025db32b8 P c15e23a34059456788f9775031e27ec0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:45.531682 22259 tablet_bootstrap.cc:492] T fb0f04b72dc046c0afee40b895e17a1d P b443e68b6212450286869124684bdfff: No bootstrap required, opened a new log
I20240502 07:59:45.531806 22259 ts_tablet_manager.cc:1367] T fb0f04b72dc046c0afee40b895e17a1d P b443e68b6212450286869124684bdfff: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.000s
I20240502 07:59:45.531879 22272 tablet_bootstrap.cc:492] T fb0f04b72dc046c0afee40b895e17a1d P 35642aa2099c40e59bc2092604078e07: No bootstrap required, opened a new log
I20240502 07:59:45.531989 22272 ts_tablet_manager.cc:1367] T fb0f04b72dc046c0afee40b895e17a1d P 35642aa2099c40e59bc2092604078e07: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 07:59:45.532402 22272 raft_consensus.cc:348] T fb0f04b72dc046c0afee40b895e17a1d P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.532492 22272 raft_consensus.cc:374] T fb0f04b72dc046c0afee40b895e17a1d P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:45.532430 22259 raft_consensus.cc:348] T fb0f04b72dc046c0afee40b895e17a1d P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.532521 22272 raft_consensus.cc:729] T fb0f04b72dc046c0afee40b895e17a1d P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35642aa2099c40e59bc2092604078e07, State: Initialized, Role: FOLLOWER
I20240502 07:59:45.532579 22259 raft_consensus.cc:374] T fb0f04b72dc046c0afee40b895e17a1d P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:45.532663 22259 raft_consensus.cc:729] T fb0f04b72dc046c0afee40b895e17a1d P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b443e68b6212450286869124684bdfff, State: Initialized, Role: FOLLOWER
I20240502 07:59:45.532655 22272 consensus_queue.cc:260] T fb0f04b72dc046c0afee40b895e17a1d P 35642aa2099c40e59bc2092604078e07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.532840 22259 consensus_queue.cc:260] T fb0f04b72dc046c0afee40b895e17a1d P b443e68b6212450286869124684bdfff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.532918 22272 ts_tablet_manager.cc:1398] T fb0f04b72dc046c0afee40b895e17a1d P 35642aa2099c40e59bc2092604078e07: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:45.533071 22272 tablet_bootstrap.cc:492] T 9ad40762dc784dfa9f3507e025db32b8 P 35642aa2099c40e59bc2092604078e07: Bootstrap starting.
I20240502 07:59:45.533881 22259 ts_tablet_manager.cc:1398] T fb0f04b72dc046c0afee40b895e17a1d P b443e68b6212450286869124684bdfff: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20240502 07:59:45.533953 22272 tablet_bootstrap.cc:654] T 9ad40762dc784dfa9f3507e025db32b8 P 35642aa2099c40e59bc2092604078e07: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:45.534150 22259 tablet_bootstrap.cc:492] T 9ad40762dc784dfa9f3507e025db32b8 P b443e68b6212450286869124684bdfff: Bootstrap starting.
I20240502 07:59:45.535391 22259 tablet_bootstrap.cc:654] T 9ad40762dc784dfa9f3507e025db32b8 P b443e68b6212450286869124684bdfff: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:45.535835 22272 tablet_bootstrap.cc:492] T 9ad40762dc784dfa9f3507e025db32b8 P 35642aa2099c40e59bc2092604078e07: No bootstrap required, opened a new log
I20240502 07:59:45.535913 22272 ts_tablet_manager.cc:1367] T 9ad40762dc784dfa9f3507e025db32b8 P 35642aa2099c40e59bc2092604078e07: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:59:45.536276 22272 raft_consensus.cc:348] T 9ad40762dc784dfa9f3507e025db32b8 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.536401 22272 raft_consensus.cc:374] T 9ad40762dc784dfa9f3507e025db32b8 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:45.536440 22272 raft_consensus.cc:729] T 9ad40762dc784dfa9f3507e025db32b8 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35642aa2099c40e59bc2092604078e07, State: Initialized, Role: FOLLOWER
I20240502 07:59:45.536574 22272 consensus_queue.cc:260] T 9ad40762dc784dfa9f3507e025db32b8 P 35642aa2099c40e59bc2092604078e07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.536832 22272 ts_tablet_manager.cc:1398] T 9ad40762dc784dfa9f3507e025db32b8 P 35642aa2099c40e59bc2092604078e07: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:45.539662 22259 tablet_bootstrap.cc:492] T 9ad40762dc784dfa9f3507e025db32b8 P b443e68b6212450286869124684bdfff: No bootstrap required, opened a new log
I20240502 07:59:45.539739 22259 ts_tablet_manager.cc:1367] T 9ad40762dc784dfa9f3507e025db32b8 P b443e68b6212450286869124684bdfff: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 07:59:45.540125 22259 raft_consensus.cc:348] T 9ad40762dc784dfa9f3507e025db32b8 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.540234 22259 raft_consensus.cc:374] T 9ad40762dc784dfa9f3507e025db32b8 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:45.540305 22259 raft_consensus.cc:729] T 9ad40762dc784dfa9f3507e025db32b8 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b443e68b6212450286869124684bdfff, State: Initialized, Role: FOLLOWER
I20240502 07:59:45.540428 22259 consensus_queue.cc:260] T 9ad40762dc784dfa9f3507e025db32b8 P b443e68b6212450286869124684bdfff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.540704 22259 ts_tablet_manager.cc:1398] T 9ad40762dc784dfa9f3507e025db32b8 P b443e68b6212450286869124684bdfff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:45.557153 21926 raft_consensus.cc:482] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:45.557247 21926 raft_consensus.cc:504] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.557782 21926 leader_election.cc:288] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315), c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085), b443e68b6212450286869124684bdfff (127.3.139.69:37923), b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577), 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703), b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331)
I20240502 07:59:45.558117 20273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "574fdd7a3c114ac9928ac12d1a7a4186" candidate_uuid: "35642aa2099c40e59bc2092604078e07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" is_pre_election: true
I20240502 07:59:45.558117 20140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "574fdd7a3c114ac9928ac12d1a7a4186" candidate_uuid: "35642aa2099c40e59bc2092604078e07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7180fe4574c09a8dce7c27500aab2" is_pre_election: true
I20240502 07:59:45.558205 20539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "574fdd7a3c114ac9928ac12d1a7a4186" candidate_uuid: "35642aa2099c40e59bc2092604078e07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e951f009258e473cb57cf8773c7975d4" is_pre_election: true
I20240502 07:59:45.558251 20140 raft_consensus.cc:2454] T 574fdd7a3c114ac9928ac12d1a7a4186 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35642aa2099c40e59bc2092604078e07 in term 0.
I20240502 07:59:45.558251 20273 raft_consensus.cc:2454] T 574fdd7a3c114ac9928ac12d1a7a4186 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35642aa2099c40e59bc2092604078e07 in term 0.
I20240502 07:59:45.558337 20539 raft_consensus.cc:2454] T 574fdd7a3c114ac9928ac12d1a7a4186 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35642aa2099c40e59bc2092604078e07 in term 0.
I20240502 07:59:45.558315 20668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "574fdd7a3c114ac9928ac12d1a7a4186" candidate_uuid: "35642aa2099c40e59bc2092604078e07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b443e68b6212450286869124684bdfff" is_pre_election: true
I20240502 07:59:45.558431 20668 raft_consensus.cc:2454] T 574fdd7a3c114ac9928ac12d1a7a4186 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35642aa2099c40e59bc2092604078e07 in term 0.
I20240502 07:59:45.558605 20933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "574fdd7a3c114ac9928ac12d1a7a4186" candidate_uuid: "35642aa2099c40e59bc2092604078e07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f471f8b5944986a7d878cf6ac550b8" is_pre_election: true
I20240502 07:59:45.558749 20933 raft_consensus.cc:2454] T 574fdd7a3c114ac9928ac12d1a7a4186 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35642aa2099c40e59bc2092604078e07 in term 0.
I20240502 07:59:45.558784 20341 leader_election.cc:302] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 29a7180fe4574c09a8dce7c27500aab2, 35642aa2099c40e59bc2092604078e07, b7604c0eb3bf4a7cb260b564a94a1191, e951f009258e473cb57cf8773c7975d4; no voters:
I20240502 07:59:45.558941 21926 raft_consensus.cc:2781] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:45.558991 21926 raft_consensus.cc:482] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:45.559026 21926 raft_consensus.cc:3037] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:45.559274 20802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "574fdd7a3c114ac9928ac12d1a7a4186" candidate_uuid: "35642aa2099c40e59bc2092604078e07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15e23a34059456788f9775031e27ec0" is_pre_election: true
I20240502 07:59:45.559401 20802 raft_consensus.cc:2454] T 574fdd7a3c114ac9928ac12d1a7a4186 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35642aa2099c40e59bc2092604078e07 in term 0.
I20240502 07:59:45.560039 21926 raft_consensus.cc:504] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.560570 21926 leader_election.cc:288] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [CANDIDATE]: Term 1 election: Requested vote from peers e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315), c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085), b443e68b6212450286869124684bdfff (127.3.139.69:37923), b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577), 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703), b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331)
I20240502 07:59:45.560802 20933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "574fdd7a3c114ac9928ac12d1a7a4186" candidate_uuid: "35642aa2099c40e59bc2092604078e07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f471f8b5944986a7d878cf6ac550b8"
I20240502 07:59:45.560906 20933 raft_consensus.cc:3037] T 574fdd7a3c114ac9928ac12d1a7a4186 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:45.560885 20140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "574fdd7a3c114ac9928ac12d1a7a4186" candidate_uuid: "35642aa2099c40e59bc2092604078e07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7180fe4574c09a8dce7c27500aab2"
I20240502 07:59:45.560976 20539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "574fdd7a3c114ac9928ac12d1a7a4186" candidate_uuid: "35642aa2099c40e59bc2092604078e07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e951f009258e473cb57cf8773c7975d4"
I20240502 07:59:45.560995 20140 raft_consensus.cc:3037] T 574fdd7a3c114ac9928ac12d1a7a4186 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:45.561023 20802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "574fdd7a3c114ac9928ac12d1a7a4186" candidate_uuid: "35642aa2099c40e59bc2092604078e07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15e23a34059456788f9775031e27ec0"
I20240502 07:59:45.561059 20539 raft_consensus.cc:3037] T 574fdd7a3c114ac9928ac12d1a7a4186 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:45.561108 20802 raft_consensus.cc:3037] T 574fdd7a3c114ac9928ac12d1a7a4186 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:45.561805 20933 raft_consensus.cc:2454] T 574fdd7a3c114ac9928ac12d1a7a4186 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35642aa2099c40e59bc2092604078e07 in term 1.
I20240502 07:59:45.561940 20539 raft_consensus.cc:2454] T 574fdd7a3c114ac9928ac12d1a7a4186 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35642aa2099c40e59bc2092604078e07 in term 1.
I20240502 07:59:45.561959 20802 raft_consensus.cc:2454] T 574fdd7a3c114ac9928ac12d1a7a4186 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35642aa2099c40e59bc2092604078e07 in term 1.
I20240502 07:59:45.561959 20140 raft_consensus.cc:2454] T 574fdd7a3c114ac9928ac12d1a7a4186 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35642aa2099c40e59bc2092604078e07 in term 1.
I20240502 07:59:45.562296 20343 leader_election.cc:302] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 29a7180fe4574c09a8dce7c27500aab2, 35642aa2099c40e59bc2092604078e07, b5f471f8b5944986a7d878cf6ac550b8, c15e23a34059456788f9775031e27ec0; no voters:
I20240502 07:59:45.562707 20668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "574fdd7a3c114ac9928ac12d1a7a4186" candidate_uuid: "35642aa2099c40e59bc2092604078e07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b443e68b6212450286869124684bdfff"
I20240502 07:59:45.562741 20273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "574fdd7a3c114ac9928ac12d1a7a4186" candidate_uuid: "35642aa2099c40e59bc2092604078e07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7604c0eb3bf4a7cb260b564a94a1191"
I20240502 07:59:45.562822 20668 raft_consensus.cc:3037] T 574fdd7a3c114ac9928ac12d1a7a4186 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:45.562829 20273 raft_consensus.cc:3037] T 574fdd7a3c114ac9928ac12d1a7a4186 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:45.563491 21926 raft_consensus.cc:2781] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:45.563686 21926 raft_consensus.cc:686] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [term 1 LEADER]: Becoming Leader. State: Replica: 35642aa2099c40e59bc2092604078e07, State: Running, Role: LEADER
I20240502 07:59:45.563683 20668 raft_consensus.cc:2454] T 574fdd7a3c114ac9928ac12d1a7a4186 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35642aa2099c40e59bc2092604078e07 in term 1.
I20240502 07:59:45.563683 20273 raft_consensus.cc:2454] T 574fdd7a3c114ac9928ac12d1a7a4186 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35642aa2099c40e59bc2092604078e07 in term 1.
I20240502 07:59:45.563843 21926 consensus_queue.cc:237] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.565830 19992 catalog_manager.cc:5484] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 reported cstate change: term changed from 0 to 1, leader changed from <none> to 35642aa2099c40e59bc2092604078e07 (127.3.139.67). New cstate: current_term: 1 leader_uuid: "35642aa2099c40e59bc2092604078e07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: HEALTHY } } }
I20240502 07:59:45.577204 21954 raft_consensus.cc:482] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:45.577302 21954 raft_consensus.cc:504] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.577903 21954 leader_election.cc:288] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b443e68b6212450286869124684bdfff (127.3.139.69:37923), c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085), b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577), b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331), 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703), 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587)
I20240502 07:59:45.578168 20933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb0f04b72dc046c0afee40b895e17a1d" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f471f8b5944986a7d878cf6ac550b8" is_pre_election: true
I20240502 07:59:45.578191 20405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb0f04b72dc046c0afee40b895e17a1d" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35642aa2099c40e59bc2092604078e07" is_pre_election: true
I20240502 07:59:45.578286 20933 raft_consensus.cc:2454] T fb0f04b72dc046c0afee40b895e17a1d P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e951f009258e473cb57cf8773c7975d4 in term 0.
I20240502 07:59:45.578300 20405 raft_consensus.cc:2454] T fb0f04b72dc046c0afee40b895e17a1d P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e951f009258e473cb57cf8773c7975d4 in term 0.
I20240502 07:59:45.578397 20273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb0f04b72dc046c0afee40b895e17a1d" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" is_pre_election: true
I20240502 07:59:45.578395 20140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb0f04b72dc046c0afee40b895e17a1d" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7180fe4574c09a8dce7c27500aab2" is_pre_election: true
I20240502 07:59:45.578500 20273 raft_consensus.cc:2454] T fb0f04b72dc046c0afee40b895e17a1d P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e951f009258e473cb57cf8773c7975d4 in term 0.
I20240502 07:59:45.578511 20140 raft_consensus.cc:2454] T fb0f04b72dc046c0afee40b895e17a1d P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e951f009258e473cb57cf8773c7975d4 in term 0.
I20240502 07:59:45.578760 20474 leader_election.cc:302] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 35642aa2099c40e59bc2092604078e07, b5f471f8b5944986a7d878cf6ac550b8, b7604c0eb3bf4a7cb260b564a94a1191, e951f009258e473cb57cf8773c7975d4; no voters:
I20240502 07:59:45.578907 21954 raft_consensus.cc:2781] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:45.578902 20802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb0f04b72dc046c0afee40b895e17a1d" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15e23a34059456788f9775031e27ec0" is_pre_election: true
I20240502 07:59:45.578958 21954 raft_consensus.cc:482] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:45.578994 21954 raft_consensus.cc:3037] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:45.579007 20802 raft_consensus.cc:2454] T fb0f04b72dc046c0afee40b895e17a1d P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e951f009258e473cb57cf8773c7975d4 in term 0.
I20240502 07:59:45.579213 20668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb0f04b72dc046c0afee40b895e17a1d" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b443e68b6212450286869124684bdfff" is_pre_election: true
I20240502 07:59:45.579326 20668 raft_consensus.cc:2454] T fb0f04b72dc046c0afee40b895e17a1d P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e951f009258e473cb57cf8773c7975d4 in term 0.
I20240502 07:59:45.579917 21954 raft_consensus.cc:504] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.580379 21954 leader_election.cc:288] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [CANDIDATE]: Term 1 election: Requested vote from peers b443e68b6212450286869124684bdfff (127.3.139.69:37923), c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085), b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577), b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331), 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703), 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587)
I20240502 07:59:45.580605 20140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb0f04b72dc046c0afee40b895e17a1d" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a7180fe4574c09a8dce7c27500aab2"
I20240502 07:59:45.580652 20933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb0f04b72dc046c0afee40b895e17a1d" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f471f8b5944986a7d878cf6ac550b8"
I20240502 07:59:45.580703 20140 raft_consensus.cc:3037] T fb0f04b72dc046c0afee40b895e17a1d P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:45.580703 20668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb0f04b72dc046c0afee40b895e17a1d" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b443e68b6212450286869124684bdfff"
I20240502 07:59:45.580703 20273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb0f04b72dc046c0afee40b895e17a1d" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7604c0eb3bf4a7cb260b564a94a1191"
I20240502 07:59:45.580751 20933 raft_consensus.cc:3037] T fb0f04b72dc046c0afee40b895e17a1d P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:45.580785 20273 raft_consensus.cc:3037] T fb0f04b72dc046c0afee40b895e17a1d P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:45.580783 20668 raft_consensus.cc:3037] T fb0f04b72dc046c0afee40b895e17a1d P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:45.581549 20140 raft_consensus.cc:2454] T fb0f04b72dc046c0afee40b895e17a1d P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e951f009258e473cb57cf8773c7975d4 in term 1.
I20240502 07:59:45.581590 20273 raft_consensus.cc:2454] T fb0f04b72dc046c0afee40b895e17a1d P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e951f009258e473cb57cf8773c7975d4 in term 1.
I20240502 07:59:45.581601 20933 raft_consensus.cc:2454] T fb0f04b72dc046c0afee40b895e17a1d P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e951f009258e473cb57cf8773c7975d4 in term 1.
I20240502 07:59:45.581601 20668 raft_consensus.cc:2454] T fb0f04b72dc046c0afee40b895e17a1d P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e951f009258e473cb57cf8773c7975d4 in term 1.
I20240502 07:59:45.582010 20474 leader_election.cc:302] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 29a7180fe4574c09a8dce7c27500aab2, b443e68b6212450286869124684bdfff, b5f471f8b5944986a7d878cf6ac550b8, e951f009258e473cb57cf8773c7975d4; no voters:
I20240502 07:59:45.582054 20802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb0f04b72dc046c0afee40b895e17a1d" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15e23a34059456788f9775031e27ec0"
I20240502 07:59:45.582054 20405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb0f04b72dc046c0afee40b895e17a1d" candidate_uuid: "e951f009258e473cb57cf8773c7975d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35642aa2099c40e59bc2092604078e07"
I20240502 07:59:45.582152 20802 raft_consensus.cc:3037] T fb0f04b72dc046c0afee40b895e17a1d P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:45.582152 20405 raft_consensus.cc:3037] T fb0f04b72dc046c0afee40b895e17a1d P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:45.582177 21954 raft_consensus.cc:2781] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:45.582253 21954 raft_consensus.cc:686] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [term 1 LEADER]: Becoming Leader. State: Replica: e951f009258e473cb57cf8773c7975d4, State: Running, Role: LEADER
I20240502 07:59:45.582501 21954 consensus_queue.cc:237] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.582985 20802 raft_consensus.cc:2454] T fb0f04b72dc046c0afee40b895e17a1d P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e951f009258e473cb57cf8773c7975d4 in term 1.
I20240502 07:59:45.582985 20405 raft_consensus.cc:2454] T fb0f04b72dc046c0afee40b895e17a1d P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e951f009258e473cb57cf8773c7975d4 in term 1.
I20240502 07:59:45.584734 19992 catalog_manager.cc:5484] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to e951f009258e473cb57cf8773c7975d4 (127.3.139.68). New cstate: current_term: 1 leader_uuid: "e951f009258e473cb57cf8773c7975d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:45.677997 21794 raft_consensus.cc:482] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:45.678122 21794 raft_consensus.cc:504] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.678745 21794 leader_election.cc:288] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085), e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315), b443e68b6212450286869124684bdfff (127.3.139.69:37923), b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577), b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331), 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587)
I20240502 07:59:45.679144 20671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ad40762dc784dfa9f3507e025db32b8" candidate_uuid: "29a7180fe4574c09a8dce7c27500aab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b443e68b6212450286869124684bdfff" is_pre_election: true
I20240502 07:59:45.679183 20273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ad40762dc784dfa9f3507e025db32b8" candidate_uuid: "29a7180fe4574c09a8dce7c27500aab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" is_pre_election: true
I20240502 07:59:45.679281 20671 raft_consensus.cc:2454] T 9ad40762dc784dfa9f3507e025db32b8 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29a7180fe4574c09a8dce7c27500aab2 in term 0.
I20240502 07:59:45.679287 20273 raft_consensus.cc:2454] T 9ad40762dc784dfa9f3507e025db32b8 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29a7180fe4574c09a8dce7c27500aab2 in term 0.
I20240502 07:59:45.679421 20802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ad40762dc784dfa9f3507e025db32b8" candidate_uuid: "29a7180fe4574c09a8dce7c27500aab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15e23a34059456788f9775031e27ec0" is_pre_election: true
I20240502 07:59:45.679567 20802 raft_consensus.cc:2454] T 9ad40762dc784dfa9f3507e025db32b8 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29a7180fe4574c09a8dce7c27500aab2 in term 0.
I20240502 07:59:45.679529 20539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ad40762dc784dfa9f3507e025db32b8" candidate_uuid: "29a7180fe4574c09a8dce7c27500aab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e951f009258e473cb57cf8773c7975d4" is_pre_election: true
I20240502 07:59:45.679674 20933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ad40762dc784dfa9f3507e025db32b8" candidate_uuid: "29a7180fe4574c09a8dce7c27500aab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f471f8b5944986a7d878cf6ac550b8" is_pre_election: true
I20240502 07:59:45.679704 20539 raft_consensus.cc:2454] T 9ad40762dc784dfa9f3507e025db32b8 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29a7180fe4574c09a8dce7c27500aab2 in term 0.
I20240502 07:59:45.679796 20933 raft_consensus.cc:2454] T 9ad40762dc784dfa9f3507e025db32b8 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29a7180fe4574c09a8dce7c27500aab2 in term 0.
I20240502 07:59:45.680033 20077 leader_election.cc:302] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 29a7180fe4574c09a8dce7c27500aab2, b7604c0eb3bf4a7cb260b564a94a1191, c15e23a34059456788f9775031e27ec0, e951f009258e473cb57cf8773c7975d4; no voters:
I20240502 07:59:45.680204 21794 raft_consensus.cc:2781] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:45.680272 21794 raft_consensus.cc:482] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:45.680317 21794 raft_consensus.cc:3037] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:45.680706 20405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ad40762dc784dfa9f3507e025db32b8" candidate_uuid: "29a7180fe4574c09a8dce7c27500aab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35642aa2099c40e59bc2092604078e07" is_pre_election: true
I20240502 07:59:45.680830 20405 raft_consensus.cc:2454] T 9ad40762dc784dfa9f3507e025db32b8 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29a7180fe4574c09a8dce7c27500aab2 in term 0.
I20240502 07:59:45.681304 21794 raft_consensus.cc:504] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.681823 21794 leader_election.cc:288] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [CANDIDATE]: Term 1 election: Requested vote from peers c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085), e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315), b443e68b6212450286869124684bdfff (127.3.139.69:37923), b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577), b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331), 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587)
I20240502 07:59:45.682000 20539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ad40762dc784dfa9f3507e025db32b8" candidate_uuid: "29a7180fe4574c09a8dce7c27500aab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e951f009258e473cb57cf8773c7975d4"
I20240502 07:59:45.682098 20539 raft_consensus.cc:3037] T 9ad40762dc784dfa9f3507e025db32b8 P e951f009258e473cb57cf8773c7975d4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:45.682332 20802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ad40762dc784dfa9f3507e025db32b8" candidate_uuid: "29a7180fe4574c09a8dce7c27500aab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15e23a34059456788f9775031e27ec0"
I20240502 07:59:45.682427 20802 raft_consensus.cc:3037] T 9ad40762dc784dfa9f3507e025db32b8 P c15e23a34059456788f9775031e27ec0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:45.682929 20273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ad40762dc784dfa9f3507e025db32b8" candidate_uuid: "29a7180fe4574c09a8dce7c27500aab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7604c0eb3bf4a7cb260b564a94a1191"
I20240502 07:59:45.682996 20539 raft_consensus.cc:2454] T 9ad40762dc784dfa9f3507e025db32b8 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29a7180fe4574c09a8dce7c27500aab2 in term 1.
I20240502 07:59:45.683032 20273 raft_consensus.cc:3037] T 9ad40762dc784dfa9f3507e025db32b8 P b7604c0eb3bf4a7cb260b564a94a1191 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:45.683755 20273 raft_consensus.cc:2454] T 9ad40762dc784dfa9f3507e025db32b8 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29a7180fe4574c09a8dce7c27500aab2 in term 1.
I20240502 07:59:45.683784 20802 raft_consensus.cc:2454] T 9ad40762dc784dfa9f3507e025db32b8 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29a7180fe4574c09a8dce7c27500aab2 in term 1.
I20240502 07:59:45.684036 20077 leader_election.cc:302] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 29a7180fe4574c09a8dce7c27500aab2, b7604c0eb3bf4a7cb260b564a94a1191, c15e23a34059456788f9775031e27ec0, e951f009258e473cb57cf8773c7975d4; no voters:
I20240502 07:59:45.684198 21794 raft_consensus.cc:2781] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:45.684206 20405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ad40762dc784dfa9f3507e025db32b8" candidate_uuid: "29a7180fe4574c09a8dce7c27500aab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35642aa2099c40e59bc2092604078e07"
I20240502 07:59:45.684293 20405 raft_consensus.cc:3037] T 9ad40762dc784dfa9f3507e025db32b8 P 35642aa2099c40e59bc2092604078e07 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:45.684409 21794 raft_consensus.cc:686] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 LEADER]: Becoming Leader. State: Replica: 29a7180fe4574c09a8dce7c27500aab2, State: Running, Role: LEADER
I20240502 07:59:45.684552 21794 consensus_queue.cc:237] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:45.685082 20405 raft_consensus.cc:2454] T 9ad40762dc784dfa9f3507e025db32b8 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29a7180fe4574c09a8dce7c27500aab2 in term 1.
I20240502 07:59:45.685412 20671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ad40762dc784dfa9f3507e025db32b8" candidate_uuid: "29a7180fe4574c09a8dce7c27500aab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b443e68b6212450286869124684bdfff"
I20240502 07:59:45.685515 20671 raft_consensus.cc:3037] T 9ad40762dc784dfa9f3507e025db32b8 P b443e68b6212450286869124684bdfff [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:45.686426 20671 raft_consensus.cc:2454] T 9ad40762dc784dfa9f3507e025db32b8 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29a7180fe4574c09a8dce7c27500aab2 in term 1.
I20240502 07:59:45.687077 19994 catalog_manager.cc:5484] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65). New cstate: current_term: 1 leader_uuid: "29a7180fe4574c09a8dce7c27500aab2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:45.687271 20933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ad40762dc784dfa9f3507e025db32b8" candidate_uuid: "29a7180fe4574c09a8dce7c27500aab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f471f8b5944986a7d878cf6ac550b8"
I20240502 07:59:45.687383 20933 raft_consensus.cc:3037] T 9ad40762dc784dfa9f3507e025db32b8 P b5f471f8b5944986a7d878cf6ac550b8 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:45.688225 20933 raft_consensus.cc:2454] T 9ad40762dc784dfa9f3507e025db32b8 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29a7180fe4574c09a8dce7c27500aab2 in term 1.
W20240502 07:59:45.838021 22287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:45.838209 22287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:45.841220 22287 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_6'
I20240502 07:59:45.923978 20273 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:45.924527 20140 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:45.924552 20802 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:45.924921 22224 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:45.925221 22224 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:45.926029 20671 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:45.926744 20933 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:45.926841 20407 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:45.926841 20273 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:45.927198 21794 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:45.928928 20275 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:45.929236 20933 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:45.929611 21926 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:45.929920 21926 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:45.930076 20801 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:45.930091 20143 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:45.930423 21926 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:45.931566 22282 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:45.934046 20539 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:45.934576 22282 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:45.936564 20671 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:45.938176 21954 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:45.938464 21954 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:45.938721 21954 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:45.939026 21954 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:45.939715 20800 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:45.940060 20537 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:45.940218 22285 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:45.942215 20671 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:45.942808 20935 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:45.943318 22285 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:45.943946 22285 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:45.947585 21926 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:45.949193 20407 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:45.951400 22285 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:45.953804 22285 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:45.978132 22351 mvcc.cc:205] Tried to move back new op lower bound from 7023152275202629632 to 7023152274167328768. Current Snapshot: MvccSnapshot[applied={T|T < 7023152275153199104}]
I20240502 07:59:46.054744 22358 mvcc.cc:205] Tried to move back new op lower bound from 7023152275202629632 to 7023152274167328768. Current Snapshot: MvccSnapshot[applied={T|T < 7023152275153199104}]
INSERT report
rows total: 2000
time total: 200.136 ms
time per row: 0.100068 ms
I20240502 07:59:46.083809 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.73:38741
--local_ip_for_outbound_sockets=127.3.139.73
--tserver_master_addrs=127.3.139.126:42555
--webserver_port=35637
--webserver_interface=127.3.139.73
--builtin_ntp_servers=127.3.139.84:37307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240502 07:59:46.188534 22392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:46.188760 22392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:59:46.188820 22392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:46.191859 22392 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:59:46.191915 22392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:59:46.191992 22392 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.73
I20240502 07:59:46.195531 22392 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:37307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.73:38741
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/data/info.pb
--webserver_interface=127.3.139.73
--webserver_port=35637
--tserver_master_addrs=127.3.139.126:42555
--never_fsync=true
--heap_profile_path=/tmp/kudu.22392
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.73
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:59:46.196314 22392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:59:46.197358 22392 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:59:46.206276 22398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:46.206593 22397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:46.206938 22400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:46.207374 22392 server_base.cc:1034] running on GCE node
I20240502 07:59:46.207711 22392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:59:46.208264 22392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:59:46.209481 22392 hybrid_clock.cc:648] HybridClock initialized: now 1714636786209453 us; error 37 us; skew 500 ppm
I20240502 07:59:46.211441 22392 webserver.cc:460] Webserver started at http://127.3.139.73:35637/ using document root <none> and password file <none>
I20240502 07:59:46.212039 22392 fs_manager.cc:359] Metadata directory not provided
I20240502 07:59:46.212131 22392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:59:46.216315 22392 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:46.218480 22407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:46.219868 22392 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240502 07:59:46.219954 22392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/wal
uuid: "bfb1b3d596ef474faa229a835329cf20"
format_stamp: "Formatted at 2024-05-02 07:59:37 on dist-test-slave-twcr"
I20240502 07:59:46.220315 22392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:59:46.236001 22392 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:59:46.236524 22392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:59:46.236673 22392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:59:46.237145 22392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:59:46.237931 22392 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:59:46.237972 22392 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:46.238021 22392 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:59:46.238049 22392 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:46.245523 22392 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.73:38741
I20240502 07:59:46.245566 22521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.73:38741 every 8 connection(s)
I20240502 07:59:46.246415 22392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-8/data/info.pb
I20240502 07:59:46.248693 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 22392
I20240502 07:59:46.249114 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-9/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-9/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-9/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-9/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.74:45559
--local_ip_for_outbound_sockets=127.3.139.74
--tserver_master_addrs=127.3.139.126:42555
--webserver_port=45863
--webserver_interface=127.3.139.74
--builtin_ntp_servers=127.3.139.84:37307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 07:59:46.254720 22522 heartbeater.cc:347] Connected to a master server at 127.3.139.126:42555
I20240502 07:59:46.254902 22522 heartbeater.cc:464] Registering TS with master...
I20240502 07:59:46.255432 22522 heartbeater.cc:511] Master 127.3.139.126:42555 requested a full tablet report, sending...
I20240502 07:59:46.256621 19993 ts_manager.cc:194] Re-registered known tserver with Master: bfb1b3d596ef474faa229a835329cf20 (127.3.139.73:38741)
I20240502 07:59:46.257362 19993 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.73:54661
W20240502 07:59:46.354727 22525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:46.354894 22525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:59:46.354949 22525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:46.358397 22525 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:59:46.358462 22525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:59:46.358553 22525 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.74
I20240502 07:59:46.361950 22525 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:37307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-9/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-9/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.74:45559
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-9/data/info.pb
--webserver_interface=127.3.139.74
--webserver_port=45863
--tserver_master_addrs=127.3.139.126:42555
--never_fsync=true
--heap_profile_path=/tmp/kudu.22525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.74
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-9/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:59:46.362665 22525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:59:46.363680 22525 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:59:46.371043 22531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:46.371351 22530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:46.371677 22533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:46.372151 22525 server_base.cc:1034] running on GCE node
I20240502 07:59:46.372475 22525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:59:46.372972 22525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:59:46.374114 22525 hybrid_clock.cc:648] HybridClock initialized: now 1714636786374098 us; error 32 us; skew 500 ppm
I20240502 07:59:46.375926 22525 webserver.cc:460] Webserver started at http://127.3.139.74:45863/ using document root <none> and password file <none>
I20240502 07:59:46.376487 22525 fs_manager.cc:359] Metadata directory not provided
I20240502 07:59:46.376565 22525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:59:46.379647 22525 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20240502 07:59:46.381634 22539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:46.382776 22525 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:46.382879 22525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-9/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-9/wal
uuid: "ff9f2ccccdfa48a1afbc0e371f41c662"
format_stamp: "Formatted at 2024-05-02 07:59:37 on dist-test-slave-twcr"
I20240502 07:59:46.383278 22525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-9/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-9/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-9/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:59:46.399821 22525 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:59:46.400306 22525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:59:46.400439 22525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:59:46.400897 22525 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:59:46.401690 22525 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:59:46.401736 22525 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:46.401782 22525 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:59:46.401809 22525 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:46.410373 22525 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.74:45559
I20240502 07:59:46.410405 22654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.74:45559 every 8 connection(s)
I20240502 07:59:46.411307 22525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-9/data/info.pb
I20240502 07:59:46.414166 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 22525
I20240502 07:59:46.414664 3629 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
/tmp/dist-test-taskcXDMw5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.139.72:43861
--local_ip_for_outbound_sockets=127.3.139.72
--tserver_master_addrs=127.3.139.126:42555
--webserver_port=46689
--webserver_interface=127.3.139.72
--builtin_ntp_servers=127.3.139.84:37307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 07:59:46.420545 22655 heartbeater.cc:347] Connected to a master server at 127.3.139.126:42555
I20240502 07:59:46.420750 22655 heartbeater.cc:464] Registering TS with master...
I20240502 07:59:46.421351 22655 heartbeater.cc:511] Master 127.3.139.126:42555 requested a full tablet report, sending...
I20240502 07:59:46.422369 19993 ts_manager.cc:194] Re-registered known tserver with Master: ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559)
I20240502 07:59:46.423085 19993 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.74:59911
W20240502 07:59:46.522457 22658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:46.522630 22658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:59:46.522694 22658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:46.525841 22658 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 07:59:46.525899 22658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:59:46.525976 22658 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.139.72
I20240502 07:59:46.529506 22658 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.139.84:37307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.139.72:43861
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.3.139.72
--webserver_port=46689
--tserver_master_addrs=127.3.139.126:42555
--never_fsync=true
--heap_profile_path=/tmp/kudu.22658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.139.72
--log_dir=/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:59:46.530272 22658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:59:46.531366 22658 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:59:46.539207 22666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:46.539531 22664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:46.539884 22663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:46.540323 22658 server_base.cc:1034] running on GCE node
I20240502 07:59:46.540766 22658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:59:46.541335 22658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:59:46.542479 22658 hybrid_clock.cc:648] HybridClock initialized: now 1714636786542468 us; error 27 us; skew 500 ppm
I20240502 07:59:46.544252 22658 webserver.cc:460] Webserver started at http://127.3.139.72:46689/ using document root <none> and password file <none>
I20240502 07:59:46.544754 22658 fs_manager.cc:359] Metadata directory not provided
I20240502 07:59:46.544822 22658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:59:46.548177 22658 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240502 07:59:46.549940 22673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:46.550828 22658 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:46.550921 22658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "f014c4ad4c1342219a1004181ac33ed8"
format_stamp: "Formatted at 2024-05-02 07:59:37 on dist-test-slave-twcr"
I20240502 07:59:46.551301 22658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:59:46.585865 22658 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:59:46.586457 22658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:59:46.586594 22658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:59:46.587082 22658 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:59:46.587864 22658 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:59:46.587916 22658 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:46.587965 22658 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:59:46.587988 22658 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:46.596338 22658 rpc_server.cc:307] RPC server started. Bound to: 127.3.139.72:43861
I20240502 07:59:46.596434 22787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.139.72:43861 every 8 connection(s)
I20240502 07:59:46.597354 22658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240502 07:59:46.600905 3629 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcXDMw5/build/debug/bin/kudu as pid 22658
I20240502 07:59:46.606140 22788 heartbeater.cc:347] Connected to a master server at 127.3.139.126:42555
I20240502 07:59:46.606355 22788 heartbeater.cc:464] Registering TS with master...
I20240502 07:59:46.606920 22788 heartbeater.cc:511] Master 127.3.139.126:42555 requested a full tablet report, sending...
I20240502 07:59:46.608050 19993 ts_manager.cc:194] Re-registered known tserver with Master: f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861)
I20240502 07:59:46.608657 19993 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.139.72:36283
I20240502 07:59:46.787804 20649 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:59:46.788820 20386 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:59:46.793524 20915 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:59:46.793818 22456 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:46.795595 20253 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:59:46.808568 20517 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:59:46.810245 22588 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:46.811096 20781 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:59:46.834445 20122 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:59:46.848018 22721 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:46.941745 20122 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:59:46.951234 20781 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:59:46.951877 20517 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:59:46.951982 20649 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:59:46.955566 20386 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:59:46.955951 22456 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:46.964704 20915 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:59:46.965847 22721 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:46.969231 22588 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:46.975033 20253 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:59:47.123735 20915 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:59:47.124522 20386 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:59:47.126616 22456 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:47.128288 20649 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:59:47.128671 20253 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:59:47.143600 20781 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:59:47.151561 20122 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:59:47.164559 22588 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:47.169036 20517 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:59:47.174301 22721 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:47.217895 20406 consensus_queue.cc:237] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } }
I20240502 07:59:47.219442 20142 raft_consensus.cc:1261] T 04659b5e3504411987dafc49a5fa5ea7 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.219437 20802 raft_consensus.cc:1261] T 04659b5e3504411987dafc49a5fa5ea7 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.219583 20933 raft_consensus.cc:1261] T 04659b5e3504411987dafc49a5fa5ea7 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.219552 20537 raft_consensus.cc:1261] T 04659b5e3504411987dafc49a5fa5ea7 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.219813 22937 consensus_queue.cc:1035] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.219959 22332 consensus_queue.cc:1035] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.220124 20275 raft_consensus.cc:1261] T 04659b5e3504411987dafc49a5fa5ea7 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.220222 20668 raft_consensus.cc:1261] T 04659b5e3504411987dafc49a5fa5ea7 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.220481 22938 consensus_queue.cc:1035] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.221073 22939 consensus_queue.cc:1035] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.221422 22332 consensus_queue.cc:1035] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.221729 22937 consensus_queue.cc:1035] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.222347 22937 raft_consensus.cc:2932] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } }
I20240502 07:59:47.222800 20537 raft_consensus.cc:2932] T 04659b5e3504411987dafc49a5fa5ea7 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } }
I20240502 07:59:47.224658 20142 raft_consensus.cc:2932] T 04659b5e3504411987dafc49a5fa5ea7 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } }
I20240502 07:59:47.225018 20802 raft_consensus.cc:2932] T 04659b5e3504411987dafc49a5fa5ea7 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } }
I20240502 07:59:47.226078 20275 raft_consensus.cc:2932] T 04659b5e3504411987dafc49a5fa5ea7 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } }
I20240502 07:59:47.226253 20933 raft_consensus.cc:2932] T 04659b5e3504411987dafc49a5fa5ea7 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } }
I20240502 07:59:47.226835 19997 catalog_manager.cc:5484] T 04659b5e3504411987dafc49a5fa5ea7 P e951f009258e473cb57cf8773c7975d4 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "35642aa2099c40e59bc2092604078e07" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } }
I20240502 07:59:47.229091 20668 raft_consensus.cc:2932] T 04659b5e3504411987dafc49a5fa5ea7 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } }
I20240502 07:59:47.235941 20406 consensus_queue.cc:237] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } }
I20240502 07:59:47.237741 20142 raft_consensus.cc:1261] T 04659b5e3504411987dafc49a5fa5ea7 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.238111 20802 raft_consensus.cc:1261] T 04659b5e3504411987dafc49a5fa5ea7 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.238188 20275 raft_consensus.cc:1261] T 04659b5e3504411987dafc49a5fa5ea7 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.238382 20537 raft_consensus.cc:1261] T 04659b5e3504411987dafc49a5fa5ea7 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.238907 20933 raft_consensus.cc:1261] T 04659b5e3504411987dafc49a5fa5ea7 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.239110 22939 consensus_queue.cc:1035] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.239302 20668 raft_consensus.cc:1261] T 04659b5e3504411987dafc49a5fa5ea7 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.239254 20406 consensus_queue.cc:237] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:47.239372 22939 consensus_queue.cc:1035] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.239507 22958 consensus_queue.cc:1035] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.240967 20933 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.241150 20275 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.241395 20668 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.241530 20537 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.241539 20142 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.241904 20802 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.241848 22939 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.242571 22963 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.242623 22940 consensus_queue.cc:1035] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.242713 22958 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.242936 22964 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.243304 22937 consensus_queue.cc:1035] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.244879 20142 raft_consensus.cc:2932] T 04659b5e3504411987dafc49a5fa5ea7 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:47.244928 22963 raft_consensus.cc:2932] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:47.245584 22332 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.246284 22957 consensus_queue.cc:1035] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.246482 19988 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 04659b5e3504411987dafc49a5fa5ea7 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240502 07:59:47.247002 20933 raft_consensus.cc:2932] T 04659b5e3504411987dafc49a5fa5ea7 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:47.247862 19995 catalog_manager.cc:5484] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 reported cstate change: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New cstate: current_term: 1 leader_uuid: "35642aa2099c40e59bc2092604078e07" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:47.249032 20802 raft_consensus.cc:2932] T 04659b5e3504411987dafc49a5fa5ea7 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:47.249370 20539 raft_consensus.cc:2932] T 04659b5e3504411987dafc49a5fa5ea7 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
W20240502 07:59:47.250324 20342 consensus_peers.cc:485] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 04659b5e3504411987dafc49a5fa5ea7. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:47.251220 20273 raft_consensus.cc:2932] T 04659b5e3504411987dafc49a5fa5ea7 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:47.252770 22965 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.253305 20670 raft_consensus.cc:2932] T 04659b5e3504411987dafc49a5fa5ea7 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:47.257737 22957 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:47.258038 20668 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:47.259212 22522 heartbeater.cc:503] Master 127.3.139.126:42555 was elected leader, sending a full tablet report...
I20240502 07:59:47.259518 20537 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:47.259769 20935 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:47.261404 20001 catalog_manager.cc:5484] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "35642aa2099c40e59bc2092604078e07" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: HEALTHY } } }
I20240502 07:59:47.263698 20142 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:47.264254 20275 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:47.266110 20802 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:47.268193 20406 consensus_queue.cc:237] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } }
I20240502 07:59:47.269685 20668 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.269953 20935 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.270032 20537 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.270331 22940 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.270601 20802 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.270668 22940 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.270924 22938 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.271095 22937 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.272905 20275 consensus_queue.cc:237] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:47.273063 20142 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.273380 22965 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.273430 20273 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.273933 22965 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.275022 20537 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.274966 22965 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:47.275599 20933 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.275709 20668 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 07:59:47.276103 20340 consensus_peers.cc:485] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer bfb1b3d596ef474faa229a835329cf20 (127.3.139.73:38741): Couldn't send request to peer bfb1b3d596ef474faa229a835329cf20. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e61fe3cb61c240f9b2c898aff1951144. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:47.276070 22183 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.276311 20800 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.276381 22183 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.276636 19988 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet e61fe3cb61c240f9b2c898aff1951144 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240502 07:59:47.277174 20668 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:47.277165 20143 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:47.277235 20539 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:47.277796 22183 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.278918 20406 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.279883 19992 catalog_manager.cc:5484] T e61fe3cb61c240f9b2c898aff1951144 P 29a7180fe4574c09a8dce7c27500aab2 reported cstate change: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New cstate: current_term: 1 leader_uuid: "35642aa2099c40e59bc2092604078e07" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:47.280133 22183 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.280539 20935 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:47.281158 20142 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.281625 20802 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:47.282810 20275 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:47.283552 22183 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.283852 22183 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.285756 22980 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:47.286564 20537 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:47.286677 20142 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:47.287848 20935 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:47.288934 19992 catalog_manager.cc:5484] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:47.289106 20406 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:47.291332 20670 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:47.292826 20802 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:47.293650 20275 consensus_queue.cc:237] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } }
I20240502 07:59:47.295946 20670 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.296201 20802 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.296463 20537 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.296813 22981 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.297124 20935 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.297326 20142 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.297611 20405 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.297677 22183 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.297967 22183 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.298199 22982 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.298523 22183 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.299028 22979 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.300886 22183 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:47.301371 20405 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:47.301595 20800 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:47.302565 19987 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 0e2908de10684c84af542f16ff8727cb with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240502 07:59:47.303092 20670 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:47.303587 20537 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:47.303951 19999 catalog_manager.cc:5484] T 0e2908de10684c84af542f16ff8727cb P 35642aa2099c40e59bc2092604078e07 reported cstate change: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New cstate: current_term: 1 leader_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:47.306176 20142 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:47.306393 20935 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
W20240502 07:59:47.310441 20208 consensus_peers.cc:485] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer bfb1b3d596ef474faa229a835329cf20 (127.3.139.73:38741): Couldn't send request to peer bfb1b3d596ef474faa229a835329cf20. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e2908de10684c84af542f16ff8727cb. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:47.311338 20405 consensus_queue.cc:237] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:47.312655 20275 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.312995 22963 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.313133 20670 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.313179 20935 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.313526 22965 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.313686 20800 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.313803 22965 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.314023 22332 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.314442 20537 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.314808 22332 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.315627 20142 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.316033 22332 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.316687 20670 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:47.317174 22961 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:47.317677 20537 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:47.318009 20800 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:47.318444 20142 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:47.319834 19993 catalog_manager.cc:5484] T 574fdd7a3c114ac9928ac12d1a7a4186 P b443e68b6212450286869124684bdfff reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "35642aa2099c40e59bc2092604078e07" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:47.323827 20275 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:47.324362 20935 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:47.326685 20405 consensus_queue.cc:237] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } }
I20240502 07:59:47.328703 20537 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.328718 20670 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.329067 22963 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.329372 22963 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.329466 20142 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20240502 07:59:47.329768 20342 consensus_peers.cc:485] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 574fdd7a3c114ac9928ac12d1a7a4186. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:47.330408 22965 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.330739 20800 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.331631 22938 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.332715 22938 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:47.333225 20143 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:47.334102 19988 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 574fdd7a3c114ac9928ac12d1a7a4186 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240502 07:59:47.335294 20935 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.335391 20802 consensus_queue.cc:237] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:47.335736 22332 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.335700 19999 catalog_manager.cc:5484] T 574fdd7a3c114ac9928ac12d1a7a4186 P 29a7180fe4574c09a8dce7c27500aab2 reported cstate change: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New cstate: current_term: 1 leader_uuid: "35642aa2099c40e59bc2092604078e07" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:47.336741 20935 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:47.337023 20933 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.337826 20273 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.337855 22368 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.338258 22332 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.338362 20142 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.338353 20537 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.338734 22368 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.338922 20275 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.338937 23011 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.339391 20273 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:47.341174 23013 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.341616 20668 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:47.343262 20405 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.344216 20539 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:47.345697 20670 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.346029 23011 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.346227 20802 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:47.346359 23011 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.349496 23013 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:47.350510 20670 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:47.350713 20405 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:47.351822 20143 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:47.352947 19999 catalog_manager.cc:5484] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "c15e23a34059456788f9775031e27ec0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:47.354964 20273 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:47.356870 20933 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:47.358127 20802 consensus_queue.cc:237] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } }
I20240502 07:59:47.359822 20143 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.360369 20933 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.360478 20405 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.360679 22368 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.360913 20537 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.360905 23013 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.361078 22368 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.361666 23011 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.361918 20670 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.362226 23011 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.362309 20273 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.362318 20537 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:47.363153 23012 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.364666 22368 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:47.365092 20537 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:47.365099 20933 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:47.366832 20143 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:47.367713 19993 catalog_manager.cc:5484] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 reported cstate change: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New cstate: current_term: 1 leader_uuid: "c15e23a34059456788f9775031e27ec0" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:47.368821 19986 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 331368e52904494fa5ecf1ef594821fc with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240502 07:59:47.369380 20273 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:47.371469 20405 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:47.374615 20670 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
W20240502 07:59:47.377087 20736 consensus_peers.cc:485] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 -> Peer bfb1b3d596ef474faa229a835329cf20 (127.3.139.73:38741): Couldn't send request to peer bfb1b3d596ef474faa229a835329cf20. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 331368e52904494fa5ecf1ef594821fc. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:47.383520 20933 consensus_queue.cc:237] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:47.385157 20273 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.385629 23032 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.385803 20670 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.386626 20537 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.386695 20143 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.387166 20802 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.387341 23032 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.387657 20405 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.387704 23030 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.387854 23032 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.388135 23032 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.388291 22405 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.389482 23032 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:47.390077 20142 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:47.390440 20537 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:47.390815 20802 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:47.392457 19994 catalog_manager.cc:5484] T 1453e833d0ad494583c44b5496088f6d P 29a7180fe4574c09a8dce7c27500aab2 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "b5f471f8b5944986a7d878cf6ac550b8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:47.394409 20273 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:47.395859 20670 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:47.397576 20405 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:47.398123 20933 consensus_queue.cc:237] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } }
I20240502 07:59:47.400229 20143 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.400252 20273 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.400548 23030 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.400583 20670 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.401317 23045 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.401621 20537 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.401724 23045 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.402812 20802 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.403230 20405 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.404422 23030 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.405174 23044 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.405335 23033 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20240502 07:59:47.405647 20868 consensus_peers.cc:485] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer bfb1b3d596ef474faa229a835329cf20 (127.3.139.73:38741): Couldn't send request to peer bfb1b3d596ef474faa229a835329cf20. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1453e833d0ad494583c44b5496088f6d. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:47.406181 23033 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:47.407197 20405 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:47.408443 20537 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:47.409123 19994 catalog_manager.cc:5484] T 1453e833d0ad494583c44b5496088f6d P 35642aa2099c40e59bc2092604078e07 reported cstate change: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New cstate: current_term: 1 leader_uuid: "b5f471f8b5944986a7d878cf6ac550b8" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:47.410259 20143 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:47.413317 20802 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:47.415416 19988 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 1453e833d0ad494583c44b5496088f6d with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240502 07:59:47.415501 20802 consensus_queue.cc:237] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:47.416213 20273 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:47.416714 20668 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:47.417011 20143 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.417140 20933 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.417353 23021 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.417479 23012 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.418776 20273 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.419118 20405 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.419977 23011 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.420042 20537 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.420269 23011 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.420372 20668 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.420387 23021 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.421368 23011 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.424023 23011 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:47.425036 22655 heartbeater.cc:503] Master 127.3.139.126:42555 was elected leader, sending a full tablet report...
I20240502 07:59:47.425220 20143 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:47.426332 20273 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:47.426473 20933 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:47.427304 19994 catalog_manager.cc:5484] T 6a77bda800d44c4bbe1b082a5ea05b55 P 29a7180fe4574c09a8dce7c27500aab2 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "c15e23a34059456788f9775031e27ec0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:47.429452 20668 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:47.430251 20405 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:47.431979 20537 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:47.433923 20802 consensus_queue.cc:237] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } }
I20240502 07:59:47.435799 20273 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.436475 20143 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.436578 20933 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.437330 23021 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.437510 23012 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.439020 20405 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.439354 23012 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.439728 20935 consensus_queue.cc:237] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:47.440284 20668 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.440347 23012 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.440620 23012 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.441627 20142 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.441615 20539 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.442095 20537 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.442239 23033 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.442396 23012 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.442432 23032 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.443284 23012 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:47.443750 20273 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.443864 20933 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:47.444079 23032 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 07:59:47.444288 20738 consensus_peers.cc:485] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6a77bda800d44c4bbe1b082a5ea05b55. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:47.444525 20670 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.445219 20406 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.446017 20406 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:47.446116 19995 catalog_manager.cc:5484] T 6a77bda800d44c4bbe1b082a5ea05b55 P b5f471f8b5944986a7d878cf6ac550b8 reported cstate change: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New cstate: current_term: 1 leader_uuid: "c15e23a34059456788f9775031e27ec0" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:47.446259 20802 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.446168 20143 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:47.446652 23045 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.446779 19986 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 6a77bda800d44c4bbe1b082a5ea05b55 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240502 07:59:47.446971 23045 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.447221 23032 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.447368 20275 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:47.450289 20668 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:47.451373 20539 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:47.452880 23044 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:47.453966 20406 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:47.455420 20142 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:47.456619 19995 catalog_manager.cc:5484] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "b5f471f8b5944986a7d878cf6ac550b8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: HEALTHY } } }
I20240502 07:59:47.458750 20802 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:47.458757 20273 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:47.458765 20670 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:47.460500 20933 consensus_queue.cc:237] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } }
I20240502 07:59:47.461505 20539 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:47.462208 20406 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20240502 07:59:47.462719 20868 consensus_peers.cc:485] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer bfb1b3d596ef474faa229a835329cf20 (127.3.139.73:38741): Couldn't send request to peer bfb1b3d596ef474faa229a835329cf20. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f588a3ec49a24b1a9a9e19cdf8c30ca5. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:47.463027 20537 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.463275 23030 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.463474 23033 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.463533 20142 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.463524 20670 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.464018 23030 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.464056 20273 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.464263 23030 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.464645 20802 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.465936 23030 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.466085 23044 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.468196 23030 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:47.468593 20802 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:47.468824 20273 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:47.470418 19988 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet f588a3ec49a24b1a9a9e19cdf8c30ca5 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240502 07:59:47.471004 20406 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:47.471146 20142 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:47.471189 20537 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:47.473112 19995 catalog_manager.cc:5484] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P c15e23a34059456788f9775031e27ec0 reported cstate change: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New cstate: current_term: 1 leader_uuid: "b5f471f8b5944986a7d878cf6ac550b8" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:47.474778 20670 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:47.480559 20142 consensus_queue.cc:237] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:47.482331 20933 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.482407 20406 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.482651 20273 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.482762 20802 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.482726 23076 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.483027 20537 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.483000 23075 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.483371 23074 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.483546 22285 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.483611 20670 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.483712 23077 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.483907 23076 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.486289 23075 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:47.486907 20537 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:47.486951 20670 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:47.487094 20406 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:47.488646 20802 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:47.489188 20273 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:47.489498 19995 catalog_manager.cc:5484] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "29a7180fe4574c09a8dce7c27500aab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: HEALTHY } } }
I20240502 07:59:47.493577 20142 consensus_queue.cc:237] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } }
I20240502 07:59:47.493786 20933 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:47.496183 20273 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.496515 20537 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.496573 22285 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.496865 23075 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.496995 20802 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.498215 22285 consensus_queue.cc:1037] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Got LMP mismatch error from peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.499251 20406 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.499385 20933 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.500517 23093 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20240502 07:59:47.503402 20078 consensus_peers.cc:485] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9ad40762dc784dfa9f3507e025db32b8. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:47.503501 20670 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.503540 23091 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.503940 23090 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:47.504210 20668 consensus_queue.cc:237] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:47.504297 20802 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:47.504431 23091 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.505460 19986 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 9ad40762dc784dfa9f3507e025db32b8 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240502 07:59:47.505916 20406 raft_consensus.cc:1261] T 0f03a9b07a8c40bdbdf3ff714302a3ea P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.505923 20273 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:47.506299 20275 raft_consensus.cc:1261] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.506572 19999 catalog_manager.cc:5484] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 reported cstate change: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New cstate: current_term: 1 leader_uuid: "29a7180fe4574c09a8dce7c27500aab2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:47.506762 22245 consensus_queue.cc:1035] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.506798 20935 raft_consensus.cc:1261] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.507045 22245 consensus_queue.cc:1035] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.507352 22245 consensus_queue.cc:1035] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.507972 20933 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:47.508066 20405 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:47.510087 20142 raft_consensus.cc:1261] T 0f03a9b07a8c40bdbdf3ff714302a3ea P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.510380 20800 raft_consensus.cc:1261] T 0f03a9b07a8c40bdbdf3ff714302a3ea P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.510381 20537 raft_consensus.cc:1261] T 0f03a9b07a8c40bdbdf3ff714302a3ea P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:47.510888 22099 consensus_queue.cc:1035] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.511210 22099 consensus_queue.cc:1035] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.511234 20668 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:47.511504 22099 consensus_queue.cc:1035] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.514374 20537 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:47.516502 22099 raft_consensus.cc:2932] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:47.517218 20406 raft_consensus.cc:2932] T 0f03a9b07a8c40bdbdf3ff714302a3ea P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:47.518102 20273 raft_consensus.cc:2932] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:47.518507 20802 raft_consensus.cc:2932] T 0f03a9b07a8c40bdbdf3ff714302a3ea P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:47.519826 19994 catalog_manager.cc:5484] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "b443e68b6212450286869124684bdfff" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: HEALTHY } } }
I20240502 07:59:47.521229 20537 raft_consensus.cc:2932] T 0f03a9b07a8c40bdbdf3ff714302a3ea P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:47.524250 20671 consensus_queue.cc:237] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } }
I20240502 07:59:47.526902 20273 raft_consensus.cc:1261] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.527869 23097 consensus_queue.cc:1035] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.528245 20406 raft_consensus.cc:1261] T 0f03a9b07a8c40bdbdf3ff714302a3ea P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.528616 20802 raft_consensus.cc:1261] T 0f03a9b07a8c40bdbdf3ff714302a3ea P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.528939 23097 consensus_queue.cc:1035] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.529036 22245 consensus_queue.cc:1035] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.530339 20537 raft_consensus.cc:1261] T 0f03a9b07a8c40bdbdf3ff714302a3ea P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.530709 23097 consensus_queue.cc:1035] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:47.532375 22099 raft_consensus.cc:2932] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
W20240502 07:59:47.533353 20606 consensus_peers.cc:485] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0f03a9b07a8c40bdbdf3ff714302a3ea. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:47.533708 19987 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 0f03a9b07a8c40bdbdf3ff714302a3ea with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240502 07:59:47.534322 20273 raft_consensus.cc:2932] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:47.534400 20537 raft_consensus.cc:2932] T 0f03a9b07a8c40bdbdf3ff714302a3ea P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:47.534584 19995 catalog_manager.cc:5484] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff reported cstate change: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New cstate: current_term: 1 leader_uuid: "b443e68b6212450286869124684bdfff" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:47.536082 20802 raft_consensus.cc:2932] T 0f03a9b07a8c40bdbdf3ff714302a3ea P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:47.537653 20143 raft_consensus.cc:1261] T 0f03a9b07a8c40bdbdf3ff714302a3ea P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.538276 23098 consensus_queue.cc:1035] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.539487 20143 raft_consensus.cc:2932] T 0f03a9b07a8c40bdbdf3ff714302a3ea P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:47.540444 20406 raft_consensus.cc:2932] T 0f03a9b07a8c40bdbdf3ff714302a3ea P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:47.540831 20143 raft_consensus.cc:2932] T 0f03a9b07a8c40bdbdf3ff714302a3ea P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:47.544790 20933 raft_consensus.cc:1261] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:47.545357 23099 consensus_queue.cc:1035] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:47.546049 20933 raft_consensus.cc:2932] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:47.547458 20933 raft_consensus.cc:2932] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:47.610950 22788 heartbeater.cc:503] Master 127.3.139.126:42555 was elected leader, sending a full tablet report...
I20240502 07:59:47.814119 23179 ts_tablet_manager.cc:902] T 04659b5e3504411987dafc49a5fa5ea7 P f014c4ad4c1342219a1004181ac33ed8: Initiating tablet copy from peer 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587)
I20240502 07:59:47.814903 23179 tablet_copy_client.cc:308] T 04659b5e3504411987dafc49a5fa5ea7 P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.67:45587
I20240502 07:59:47.819684 20427 tablet_copy_service.cc:140] P 35642aa2099c40e59bc2092604078e07: Received BeginTabletCopySession request for tablet 04659b5e3504411987dafc49a5fa5ea7 from peer f014c4ad4c1342219a1004181ac33ed8 ({username='slave'} at 127.3.139.72:45177)
I20240502 07:59:47.819801 20427 tablet_copy_service.cc:161] P 35642aa2099c40e59bc2092604078e07: Beginning new tablet copy session on tablet 04659b5e3504411987dafc49a5fa5ea7 from peer f014c4ad4c1342219a1004181ac33ed8 at {username='slave'} at 127.3.139.72:45177: session id = f014c4ad4c1342219a1004181ac33ed8-04659b5e3504411987dafc49a5fa5ea7
I20240502 07:59:47.821156 20427 tablet_copy_source_session.cc:205] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:47.824909 23179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04659b5e3504411987dafc49a5fa5ea7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:47.831250 23179 tablet_copy_client.cc:783] T 04659b5e3504411987dafc49a5fa5ea7 P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:47.831450 23179 tablet_copy_client.cc:647] T 04659b5e3504411987dafc49a5fa5ea7 P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:47.833173 23179 tablet_copy_client.cc:523] T 04659b5e3504411987dafc49a5fa5ea7 P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:47.834821 23179 tablet_bootstrap.cc:492] T 04659b5e3504411987dafc49a5fa5ea7 P f014c4ad4c1342219a1004181ac33ed8: Bootstrap starting.
I20240502 07:59:47.837235 23205 ts_tablet_manager.cc:902] T 0e2908de10684c84af542f16ff8727cb P bfb1b3d596ef474faa229a835329cf20: Initiating tablet copy from peer b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331)
I20240502 07:59:47.838282 23205 tablet_copy_client.cc:308] T 0e2908de10684c84af542f16ff8727cb P bfb1b3d596ef474faa229a835329cf20: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.66:45331
I20240502 07:59:47.839298 23206 ts_tablet_manager.cc:902] T 6a77bda800d44c4bbe1b082a5ea05b55 P ff9f2ccccdfa48a1afbc0e371f41c662: Initiating tablet copy from peer c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085)
I20240502 07:59:47.840874 23206 tablet_copy_client.cc:308] T 6a77bda800d44c4bbe1b082a5ea05b55 P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.70:36085
I20240502 07:59:47.842118 23179 log.cc:824] T 04659b5e3504411987dafc49a5fa5ea7 P f014c4ad4c1342219a1004181ac33ed8: Log is configured to *not* fsync() on all Append() calls
I20240502 07:59:47.843897 20295 tablet_copy_service.cc:140] P b7604c0eb3bf4a7cb260b564a94a1191: Received BeginTabletCopySession request for tablet 0e2908de10684c84af542f16ff8727cb from peer bfb1b3d596ef474faa229a835329cf20 ({username='slave'} at 127.3.139.73:36205)
I20240502 07:59:47.844029 20295 tablet_copy_service.cc:161] P b7604c0eb3bf4a7cb260b564a94a1191: Beginning new tablet copy session on tablet 0e2908de10684c84af542f16ff8727cb from peer bfb1b3d596ef474faa229a835329cf20 at {username='slave'} at 127.3.139.73:36205: session id = bfb1b3d596ef474faa229a835329cf20-0e2908de10684c84af542f16ff8727cb
I20240502 07:59:47.845289 20295 tablet_copy_source_session.cc:205] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:47.846628 20823 tablet_copy_service.cc:140] P c15e23a34059456788f9775031e27ec0: Received BeginTabletCopySession request for tablet 6a77bda800d44c4bbe1b082a5ea05b55 from peer ff9f2ccccdfa48a1afbc0e371f41c662 ({username='slave'} at 127.3.139.74:42219)
I20240502 07:59:47.846761 20823 tablet_copy_service.cc:161] P c15e23a34059456788f9775031e27ec0: Beginning new tablet copy session on tablet 6a77bda800d44c4bbe1b082a5ea05b55 from peer ff9f2ccccdfa48a1afbc0e371f41c662 at {username='slave'} at 127.3.139.74:42219: session id = ff9f2ccccdfa48a1afbc0e371f41c662-6a77bda800d44c4bbe1b082a5ea05b55
I20240502 07:59:47.848017 20823 tablet_copy_source_session.cc:205] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:47.848794 23205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e2908de10684c84af542f16ff8727cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:47.855401 23205 tablet_copy_client.cc:783] T 0e2908de10684c84af542f16ff8727cb P bfb1b3d596ef474faa229a835329cf20: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:47.855652 23205 tablet_copy_client.cc:647] T 0e2908de10684c84af542f16ff8727cb P bfb1b3d596ef474faa229a835329cf20: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:47.857023 23206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a77bda800d44c4bbe1b082a5ea05b55. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:47.860765 23205 tablet_copy_client.cc:523] T 0e2908de10684c84af542f16ff8727cb P bfb1b3d596ef474faa229a835329cf20: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:47.862402 23205 tablet_bootstrap.cc:492] T 0e2908de10684c84af542f16ff8727cb P bfb1b3d596ef474faa229a835329cf20: Bootstrap starting.
I20240502 07:59:47.866103 23206 tablet_copy_client.cc:783] T 6a77bda800d44c4bbe1b082a5ea05b55 P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:47.866322 23206 tablet_copy_client.cc:647] T 6a77bda800d44c4bbe1b082a5ea05b55 P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:47.868664 23205 log.cc:824] T 0e2908de10684c84af542f16ff8727cb P bfb1b3d596ef474faa229a835329cf20: Log is configured to *not* fsync() on all Append() calls
I20240502 07:59:47.868820 23206 tablet_copy_client.cc:523] T 6a77bda800d44c4bbe1b082a5ea05b55 P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:47.870251 23206 tablet_bootstrap.cc:492] T 6a77bda800d44c4bbe1b082a5ea05b55 P ff9f2ccccdfa48a1afbc0e371f41c662: Bootstrap starting.
I20240502 07:59:47.870828 23179 tablet_bootstrap.cc:492] T 04659b5e3504411987dafc49a5fa5ea7 P f014c4ad4c1342219a1004181ac33ed8: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:47.871181 23179 tablet_bootstrap.cc:492] T 04659b5e3504411987dafc49a5fa5ea7 P f014c4ad4c1342219a1004181ac33ed8: Bootstrap complete.
I20240502 07:59:47.871480 23179 ts_tablet_manager.cc:1367] T 04659b5e3504411987dafc49a5fa5ea7 P f014c4ad4c1342219a1004181ac33ed8: Time spent bootstrapping tablet: real 0.037s user 0.031s sys 0.002s
I20240502 07:59:47.873860 23179 raft_consensus.cc:348] T 04659b5e3504411987dafc49a5fa5ea7 P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } }
I20240502 07:59:47.874130 23179 raft_consensus.cc:729] T 04659b5e3504411987dafc49a5fa5ea7 P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f014c4ad4c1342219a1004181ac33ed8, State: Initialized, Role: LEARNER
I20240502 07:59:47.874466 23179 consensus_queue.cc:260] T 04659b5e3504411987dafc49a5fa5ea7 P f014c4ad4c1342219a1004181ac33ed8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } }
I20240502 07:59:47.875378 23179 ts_tablet_manager.cc:1398] T 04659b5e3504411987dafc49a5fa5ea7 P f014c4ad4c1342219a1004181ac33ed8: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20240502 07:59:47.876894 23206 log.cc:824] T 6a77bda800d44c4bbe1b082a5ea05b55 P ff9f2ccccdfa48a1afbc0e371f41c662: Log is configured to *not* fsync() on all Append() calls
I20240502 07:59:47.878365 20427 tablet_copy_service.cc:342] P 35642aa2099c40e59bc2092604078e07: Request end of tablet copy session f014c4ad4c1342219a1004181ac33ed8-04659b5e3504411987dafc49a5fa5ea7 received from {username='slave'} at 127.3.139.72:45177
I20240502 07:59:47.878458 20427 tablet_copy_service.cc:433] P 35642aa2099c40e59bc2092604078e07: ending tablet copy session f014c4ad4c1342219a1004181ac33ed8-04659b5e3504411987dafc49a5fa5ea7 on tablet 04659b5e3504411987dafc49a5fa5ea7 with peer f014c4ad4c1342219a1004181ac33ed8
I20240502 07:59:47.891252 23232 ts_tablet_manager.cc:902] T e61fe3cb61c240f9b2c898aff1951144 P bfb1b3d596ef474faa229a835329cf20: Initiating tablet copy from peer 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587)
I20240502 07:59:47.891824 23232 tablet_copy_client.cc:308] T e61fe3cb61c240f9b2c898aff1951144 P bfb1b3d596ef474faa229a835329cf20: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.67:45587
I20240502 07:59:47.895912 20427 tablet_copy_service.cc:140] P 35642aa2099c40e59bc2092604078e07: Received BeginTabletCopySession request for tablet e61fe3cb61c240f9b2c898aff1951144 from peer bfb1b3d596ef474faa229a835329cf20 ({username='slave'} at 127.3.139.73:60921)
I20240502 07:59:47.896023 20427 tablet_copy_service.cc:161] P 35642aa2099c40e59bc2092604078e07: Beginning new tablet copy session on tablet e61fe3cb61c240f9b2c898aff1951144 from peer bfb1b3d596ef474faa229a835329cf20 at {username='slave'} at 127.3.139.73:60921: session id = bfb1b3d596ef474faa229a835329cf20-e61fe3cb61c240f9b2c898aff1951144
I20240502 07:59:47.896737 20427 tablet_copy_source_session.cc:205] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:47.897464 23232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e61fe3cb61c240f9b2c898aff1951144. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:47.899845 23232 tablet_copy_client.cc:783] T e61fe3cb61c240f9b2c898aff1951144 P bfb1b3d596ef474faa229a835329cf20: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:47.899995 23232 tablet_copy_client.cc:647] T e61fe3cb61c240f9b2c898aff1951144 P bfb1b3d596ef474faa229a835329cf20: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:47.901006 23232 tablet_copy_client.cc:523] T e61fe3cb61c240f9b2c898aff1951144 P bfb1b3d596ef474faa229a835329cf20: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:47.902230 23205 tablet_bootstrap.cc:492] T 0e2908de10684c84af542f16ff8727cb P bfb1b3d596ef474faa229a835329cf20: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:47.902292 23232 tablet_bootstrap.cc:492] T e61fe3cb61c240f9b2c898aff1951144 P bfb1b3d596ef474faa229a835329cf20: Bootstrap starting.
I20240502 07:59:47.902546 23205 tablet_bootstrap.cc:492] T 0e2908de10684c84af542f16ff8727cb P bfb1b3d596ef474faa229a835329cf20: Bootstrap complete.
I20240502 07:59:47.902824 23205 ts_tablet_manager.cc:1367] T 0e2908de10684c84af542f16ff8727cb P bfb1b3d596ef474faa229a835329cf20: Time spent bootstrapping tablet: real 0.041s user 0.035s sys 0.000s
I20240502 07:59:47.903223 23237 ts_tablet_manager.cc:902] T 331368e52904494fa5ecf1ef594821fc P bfb1b3d596ef474faa229a835329cf20: Initiating tablet copy from peer c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085)
I20240502 07:59:47.903733 23237 tablet_copy_client.cc:308] T 331368e52904494fa5ecf1ef594821fc P bfb1b3d596ef474faa229a835329cf20: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.70:36085
I20240502 07:59:47.904688 23206 tablet_bootstrap.cc:492] T 6a77bda800d44c4bbe1b082a5ea05b55 P ff9f2ccccdfa48a1afbc0e371f41c662: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:47.905005 23206 tablet_bootstrap.cc:492] T 6a77bda800d44c4bbe1b082a5ea05b55 P ff9f2ccccdfa48a1afbc0e371f41c662: Bootstrap complete.
I20240502 07:59:47.905257 23206 ts_tablet_manager.cc:1367] T 6a77bda800d44c4bbe1b082a5ea05b55 P ff9f2ccccdfa48a1afbc0e371f41c662: Time spent bootstrapping tablet: real 0.035s user 0.027s sys 0.007s
I20240502 07:59:47.905700 23205 raft_consensus.cc:348] T 0e2908de10684c84af542f16ff8727cb P bfb1b3d596ef474faa229a835329cf20 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } }
I20240502 07:59:47.905954 23205 raft_consensus.cc:729] T 0e2908de10684c84af542f16ff8727cb P bfb1b3d596ef474faa229a835329cf20 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: bfb1b3d596ef474faa229a835329cf20, State: Initialized, Role: LEARNER
I20240502 07:59:47.906289 23205 consensus_queue.cc:260] T 0e2908de10684c84af542f16ff8727cb P bfb1b3d596ef474faa229a835329cf20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } }
I20240502 07:59:47.907123 23205 ts_tablet_manager.cc:1398] T 0e2908de10684c84af542f16ff8727cb P bfb1b3d596ef474faa229a835329cf20: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20240502 07:59:47.907538 23206 raft_consensus.cc:348] T 6a77bda800d44c4bbe1b082a5ea05b55 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } }
I20240502 07:59:47.907773 23206 raft_consensus.cc:729] T 6a77bda800d44c4bbe1b082a5ea05b55 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ff9f2ccccdfa48a1afbc0e371f41c662, State: Initialized, Role: LEARNER
I20240502 07:59:47.908080 23206 consensus_queue.cc:260] T 6a77bda800d44c4bbe1b082a5ea05b55 P ff9f2ccccdfa48a1afbc0e371f41c662 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } }
I20240502 07:59:47.908490 20823 tablet_copy_service.cc:140] P c15e23a34059456788f9775031e27ec0: Received BeginTabletCopySession request for tablet 331368e52904494fa5ecf1ef594821fc from peer bfb1b3d596ef474faa229a835329cf20 ({username='slave'} at 127.3.139.73:46715)
I20240502 07:59:47.908576 20823 tablet_copy_service.cc:161] P c15e23a34059456788f9775031e27ec0: Beginning new tablet copy session on tablet 331368e52904494fa5ecf1ef594821fc from peer bfb1b3d596ef474faa229a835329cf20 at {username='slave'} at 127.3.139.73:46715: session id = bfb1b3d596ef474faa229a835329cf20-331368e52904494fa5ecf1ef594821fc
I20240502 07:59:47.909399 20823 tablet_copy_source_session.cc:205] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:47.909821 20295 tablet_copy_service.cc:342] P b7604c0eb3bf4a7cb260b564a94a1191: Request end of tablet copy session bfb1b3d596ef474faa229a835329cf20-0e2908de10684c84af542f16ff8727cb received from {username='slave'} at 127.3.139.73:36205
I20240502 07:59:47.909912 20295 tablet_copy_service.cc:433] P b7604c0eb3bf4a7cb260b564a94a1191: ending tablet copy session bfb1b3d596ef474faa229a835329cf20-0e2908de10684c84af542f16ff8727cb on tablet 0e2908de10684c84af542f16ff8727cb with peer bfb1b3d596ef474faa229a835329cf20
I20240502 07:59:47.910305 23237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 331368e52904494fa5ecf1ef594821fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:47.910899 23206 ts_tablet_manager.cc:1398] T 6a77bda800d44c4bbe1b082a5ea05b55 P ff9f2ccccdfa48a1afbc0e371f41c662: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20240502 07:59:47.912496 20823 tablet_copy_service.cc:342] P c15e23a34059456788f9775031e27ec0: Request end of tablet copy session ff9f2ccccdfa48a1afbc0e371f41c662-6a77bda800d44c4bbe1b082a5ea05b55 received from {username='slave'} at 127.3.139.74:42219
I20240502 07:59:47.912572 20823 tablet_copy_service.cc:433] P c15e23a34059456788f9775031e27ec0: ending tablet copy session ff9f2ccccdfa48a1afbc0e371f41c662-6a77bda800d44c4bbe1b082a5ea05b55 on tablet 6a77bda800d44c4bbe1b082a5ea05b55 with peer ff9f2ccccdfa48a1afbc0e371f41c662
I20240502 07:59:47.913055 23237 tablet_copy_client.cc:783] T 331368e52904494fa5ecf1ef594821fc P bfb1b3d596ef474faa229a835329cf20: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:47.913209 23237 tablet_copy_client.cc:647] T 331368e52904494fa5ecf1ef594821fc P bfb1b3d596ef474faa229a835329cf20: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:47.914142 23237 tablet_copy_client.cc:523] T 331368e52904494fa5ecf1ef594821fc P bfb1b3d596ef474faa229a835329cf20: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:47.915165 23237 tablet_bootstrap.cc:492] T 331368e52904494fa5ecf1ef594821fc P bfb1b3d596ef474faa229a835329cf20: Bootstrap starting.
I20240502 07:59:47.930744 23232 tablet_bootstrap.cc:492] T e61fe3cb61c240f9b2c898aff1951144 P bfb1b3d596ef474faa229a835329cf20: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:47.930976 23232 tablet_bootstrap.cc:492] T e61fe3cb61c240f9b2c898aff1951144 P bfb1b3d596ef474faa229a835329cf20: Bootstrap complete.
I20240502 07:59:47.931159 23232 ts_tablet_manager.cc:1367] T e61fe3cb61c240f9b2c898aff1951144 P bfb1b3d596ef474faa229a835329cf20: Time spent bootstrapping tablet: real 0.029s user 0.027s sys 0.000s
I20240502 07:59:47.931619 23232 raft_consensus.cc:348] T e61fe3cb61c240f9b2c898aff1951144 P bfb1b3d596ef474faa229a835329cf20 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } }
I20240502 07:59:47.931752 23232 raft_consensus.cc:729] T e61fe3cb61c240f9b2c898aff1951144 P bfb1b3d596ef474faa229a835329cf20 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: bfb1b3d596ef474faa229a835329cf20, State: Initialized, Role: LEARNER
I20240502 07:59:47.931838 23232 consensus_queue.cc:260] T e61fe3cb61c240f9b2c898aff1951144 P bfb1b3d596ef474faa229a835329cf20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } }
I20240502 07:59:47.932101 23232 ts_tablet_manager.cc:1398] T e61fe3cb61c240f9b2c898aff1951144 P bfb1b3d596ef474faa229a835329cf20: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:47.932528 20427 tablet_copy_service.cc:342] P 35642aa2099c40e59bc2092604078e07: Request end of tablet copy session bfb1b3d596ef474faa229a835329cf20-e61fe3cb61c240f9b2c898aff1951144 received from {username='slave'} at 127.3.139.73:60921
I20240502 07:59:47.932600 20427 tablet_copy_service.cc:433] P 35642aa2099c40e59bc2092604078e07: ending tablet copy session bfb1b3d596ef474faa229a835329cf20-e61fe3cb61c240f9b2c898aff1951144 on tablet e61fe3cb61c240f9b2c898aff1951144 with peer bfb1b3d596ef474faa229a835329cf20
I20240502 07:59:47.939927 23237 tablet_bootstrap.cc:492] T 331368e52904494fa5ecf1ef594821fc P bfb1b3d596ef474faa229a835329cf20: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:47.940166 23237 tablet_bootstrap.cc:492] T 331368e52904494fa5ecf1ef594821fc P bfb1b3d596ef474faa229a835329cf20: Bootstrap complete.
I20240502 07:59:47.940322 23237 ts_tablet_manager.cc:1367] T 331368e52904494fa5ecf1ef594821fc P bfb1b3d596ef474faa229a835329cf20: Time spent bootstrapping tablet: real 0.025s user 0.024s sys 0.000s
I20240502 07:59:47.940652 23237 raft_consensus.cc:348] T 331368e52904494fa5ecf1ef594821fc P bfb1b3d596ef474faa229a835329cf20 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } }
I20240502 07:59:47.940788 23237 raft_consensus.cc:729] T 331368e52904494fa5ecf1ef594821fc P bfb1b3d596ef474faa229a835329cf20 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: bfb1b3d596ef474faa229a835329cf20, State: Initialized, Role: LEARNER
I20240502 07:59:47.940882 23237 consensus_queue.cc:260] T 331368e52904494fa5ecf1ef594821fc P bfb1b3d596ef474faa229a835329cf20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } }
I20240502 07:59:47.941138 23237 ts_tablet_manager.cc:1398] T 331368e52904494fa5ecf1ef594821fc P bfb1b3d596ef474faa229a835329cf20: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:47.941552 20823 tablet_copy_service.cc:342] P c15e23a34059456788f9775031e27ec0: Request end of tablet copy session bfb1b3d596ef474faa229a835329cf20-331368e52904494fa5ecf1ef594821fc received from {username='slave'} at 127.3.139.73:46715
I20240502 07:59:47.941623 20823 tablet_copy_service.cc:433] P c15e23a34059456788f9775031e27ec0: ending tablet copy session bfb1b3d596ef474faa229a835329cf20-331368e52904494fa5ecf1ef594821fc on tablet 331368e52904494fa5ecf1ef594821fc with peer bfb1b3d596ef474faa229a835329cf20
I20240502 07:59:47.947321 23179 ts_tablet_manager.cc:902] T 574fdd7a3c114ac9928ac12d1a7a4186 P f014c4ad4c1342219a1004181ac33ed8: Initiating tablet copy from peer 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587)
I20240502 07:59:47.947732 23179 tablet_copy_client.cc:308] T 574fdd7a3c114ac9928ac12d1a7a4186 P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.67:45587
I20240502 07:59:47.948159 20427 tablet_copy_service.cc:140] P 35642aa2099c40e59bc2092604078e07: Received BeginTabletCopySession request for tablet 574fdd7a3c114ac9928ac12d1a7a4186 from peer f014c4ad4c1342219a1004181ac33ed8 ({username='slave'} at 127.3.139.72:45177)
I20240502 07:59:47.948246 20427 tablet_copy_service.cc:161] P 35642aa2099c40e59bc2092604078e07: Beginning new tablet copy session on tablet 574fdd7a3c114ac9928ac12d1a7a4186 from peer f014c4ad4c1342219a1004181ac33ed8 at {username='slave'} at 127.3.139.72:45177: session id = f014c4ad4c1342219a1004181ac33ed8-574fdd7a3c114ac9928ac12d1a7a4186
I20240502 07:59:47.948859 20427 tablet_copy_source_session.cc:205] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:47.949589 23179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 574fdd7a3c114ac9928ac12d1a7a4186. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:47.951278 23179 tablet_copy_client.cc:783] T 574fdd7a3c114ac9928ac12d1a7a4186 P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:47.951406 23179 tablet_copy_client.cc:647] T 574fdd7a3c114ac9928ac12d1a7a4186 P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:47.952445 23250 ts_tablet_manager.cc:902] T 0f03a9b07a8c40bdbdf3ff714302a3ea P f014c4ad4c1342219a1004181ac33ed8: Initiating tablet copy from peer b443e68b6212450286869124684bdfff (127.3.139.69:37923)
I20240502 07:59:47.952848 23179 tablet_copy_client.cc:523] T 574fdd7a3c114ac9928ac12d1a7a4186 P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:47.952939 23250 tablet_copy_client.cc:308] T 0f03a9b07a8c40bdbdf3ff714302a3ea P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.69:37923
I20240502 07:59:47.953863 23179 tablet_bootstrap.cc:492] T 574fdd7a3c114ac9928ac12d1a7a4186 P f014c4ad4c1342219a1004181ac33ed8: Bootstrap starting.
I20240502 07:59:47.957051 20691 tablet_copy_service.cc:140] P b443e68b6212450286869124684bdfff: Received BeginTabletCopySession request for tablet 0f03a9b07a8c40bdbdf3ff714302a3ea from peer f014c4ad4c1342219a1004181ac33ed8 ({username='slave'} at 127.3.139.72:49807)
I20240502 07:59:47.957190 20691 tablet_copy_service.cc:161] P b443e68b6212450286869124684bdfff: Beginning new tablet copy session on tablet 0f03a9b07a8c40bdbdf3ff714302a3ea from peer f014c4ad4c1342219a1004181ac33ed8 at {username='slave'} at 127.3.139.72:49807: session id = f014c4ad4c1342219a1004181ac33ed8-0f03a9b07a8c40bdbdf3ff714302a3ea
I20240502 07:59:47.958539 20691 tablet_copy_source_session.cc:205] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:47.959609 23250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f03a9b07a8c40bdbdf3ff714302a3ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:47.961005 23250 tablet_copy_client.cc:783] T 0f03a9b07a8c40bdbdf3ff714302a3ea P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:47.961128 23250 tablet_copy_client.cc:647] T 0f03a9b07a8c40bdbdf3ff714302a3ea P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:47.962383 23250 tablet_copy_client.cc:523] T 0f03a9b07a8c40bdbdf3ff714302a3ea P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:47.963240 23250 tablet_bootstrap.cc:492] T 0f03a9b07a8c40bdbdf3ff714302a3ea P f014c4ad4c1342219a1004181ac33ed8: Bootstrap starting.
I20240502 07:59:47.973351 23179 tablet_bootstrap.cc:492] T 574fdd7a3c114ac9928ac12d1a7a4186 P f014c4ad4c1342219a1004181ac33ed8: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:47.973544 23179 tablet_bootstrap.cc:492] T 574fdd7a3c114ac9928ac12d1a7a4186 P f014c4ad4c1342219a1004181ac33ed8: Bootstrap complete.
I20240502 07:59:47.973680 23179 ts_tablet_manager.cc:1367] T 574fdd7a3c114ac9928ac12d1a7a4186 P f014c4ad4c1342219a1004181ac33ed8: Time spent bootstrapping tablet: real 0.020s user 0.018s sys 0.000s
I20240502 07:59:47.974079 23179 raft_consensus.cc:348] T 574fdd7a3c114ac9928ac12d1a7a4186 P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } }
I20240502 07:59:47.974176 23179 raft_consensus.cc:729] T 574fdd7a3c114ac9928ac12d1a7a4186 P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f014c4ad4c1342219a1004181ac33ed8, State: Initialized, Role: LEARNER
I20240502 07:59:47.974267 23179 consensus_queue.cc:260] T 574fdd7a3c114ac9928ac12d1a7a4186 P f014c4ad4c1342219a1004181ac33ed8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } }
I20240502 07:59:47.974491 23179 ts_tablet_manager.cc:1398] T 574fdd7a3c114ac9928ac12d1a7a4186 P f014c4ad4c1342219a1004181ac33ed8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:47.974890 20427 tablet_copy_service.cc:342] P 35642aa2099c40e59bc2092604078e07: Request end of tablet copy session f014c4ad4c1342219a1004181ac33ed8-574fdd7a3c114ac9928ac12d1a7a4186 received from {username='slave'} at 127.3.139.72:45177
I20240502 07:59:47.974969 20427 tablet_copy_service.cc:433] P 35642aa2099c40e59bc2092604078e07: ending tablet copy session f014c4ad4c1342219a1004181ac33ed8-574fdd7a3c114ac9928ac12d1a7a4186 on tablet 574fdd7a3c114ac9928ac12d1a7a4186 with peer f014c4ad4c1342219a1004181ac33ed8
I20240502 07:59:47.980654 23250 tablet_bootstrap.cc:492] T 0f03a9b07a8c40bdbdf3ff714302a3ea P f014c4ad4c1342219a1004181ac33ed8: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:47.980878 23250 tablet_bootstrap.cc:492] T 0f03a9b07a8c40bdbdf3ff714302a3ea P f014c4ad4c1342219a1004181ac33ed8: Bootstrap complete.
I20240502 07:59:47.981019 23250 ts_tablet_manager.cc:1367] T 0f03a9b07a8c40bdbdf3ff714302a3ea P f014c4ad4c1342219a1004181ac33ed8: Time spent bootstrapping tablet: real 0.018s user 0.015s sys 0.000s
I20240502 07:59:47.981372 23250 raft_consensus.cc:348] T 0f03a9b07a8c40bdbdf3ff714302a3ea P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } }
I20240502 07:59:47.981494 23250 raft_consensus.cc:729] T 0f03a9b07a8c40bdbdf3ff714302a3ea P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f014c4ad4c1342219a1004181ac33ed8, State: Initialized, Role: LEARNER
I20240502 07:59:47.981596 23250 consensus_queue.cc:260] T 0f03a9b07a8c40bdbdf3ff714302a3ea P f014c4ad4c1342219a1004181ac33ed8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } }
I20240502 07:59:47.981832 23250 ts_tablet_manager.cc:1398] T 0f03a9b07a8c40bdbdf3ff714302a3ea P f014c4ad4c1342219a1004181ac33ed8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:47.982419 20691 tablet_copy_service.cc:342] P b443e68b6212450286869124684bdfff: Request end of tablet copy session f014c4ad4c1342219a1004181ac33ed8-0f03a9b07a8c40bdbdf3ff714302a3ea received from {username='slave'} at 127.3.139.72:49807
I20240502 07:59:47.982493 20691 tablet_copy_service.cc:433] P b443e68b6212450286869124684bdfff: ending tablet copy session f014c4ad4c1342219a1004181ac33ed8-0f03a9b07a8c40bdbdf3ff714302a3ea on tablet 0f03a9b07a8c40bdbdf3ff714302a3ea with peer f014c4ad4c1342219a1004181ac33ed8
I20240502 07:59:47.992316 23237 ts_tablet_manager.cc:902] T 1453e833d0ad494583c44b5496088f6d P bfb1b3d596ef474faa229a835329cf20: Initiating tablet copy from peer b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577)
I20240502 07:59:47.992707 23237 tablet_copy_client.cc:308] T 1453e833d0ad494583c44b5496088f6d P bfb1b3d596ef474faa229a835329cf20: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.71:39577
I20240502 07:59:47.995867 20955 tablet_copy_service.cc:140] P b5f471f8b5944986a7d878cf6ac550b8: Received BeginTabletCopySession request for tablet 1453e833d0ad494583c44b5496088f6d from peer bfb1b3d596ef474faa229a835329cf20 ({username='slave'} at 127.3.139.73:43417)
I20240502 07:59:47.995983 20955 tablet_copy_service.cc:161] P b5f471f8b5944986a7d878cf6ac550b8: Beginning new tablet copy session on tablet 1453e833d0ad494583c44b5496088f6d from peer bfb1b3d596ef474faa229a835329cf20 at {username='slave'} at 127.3.139.73:43417: session id = bfb1b3d596ef474faa229a835329cf20-1453e833d0ad494583c44b5496088f6d
I20240502 07:59:47.996989 20955 tablet_copy_source_session.cc:205] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:47.997979 23237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1453e833d0ad494583c44b5496088f6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:48.000228 23237 tablet_copy_client.cc:783] T 1453e833d0ad494583c44b5496088f6d P bfb1b3d596ef474faa229a835329cf20: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:48.000365 23237 tablet_copy_client.cc:647] T 1453e833d0ad494583c44b5496088f6d P bfb1b3d596ef474faa229a835329cf20: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:48.001660 23237 tablet_copy_client.cc:523] T 1453e833d0ad494583c44b5496088f6d P bfb1b3d596ef474faa229a835329cf20: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:48.002561 23237 tablet_bootstrap.cc:492] T 1453e833d0ad494583c44b5496088f6d P bfb1b3d596ef474faa229a835329cf20: Bootstrap starting.
I20240502 07:59:48.008987 23250 ts_tablet_manager.cc:902] T 9ad40762dc784dfa9f3507e025db32b8 P f014c4ad4c1342219a1004181ac33ed8: Initiating tablet copy from peer 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703)
I20240502 07:59:48.009373 23250 tablet_copy_client.cc:308] T 9ad40762dc784dfa9f3507e025db32b8 P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.65:44703
I20240502 07:59:48.012862 20163 tablet_copy_service.cc:140] P 29a7180fe4574c09a8dce7c27500aab2: Received BeginTabletCopySession request for tablet 9ad40762dc784dfa9f3507e025db32b8 from peer f014c4ad4c1342219a1004181ac33ed8 ({username='slave'} at 127.3.139.72:51701)
I20240502 07:59:48.013015 20163 tablet_copy_service.cc:161] P 29a7180fe4574c09a8dce7c27500aab2: Beginning new tablet copy session on tablet 9ad40762dc784dfa9f3507e025db32b8 from peer f014c4ad4c1342219a1004181ac33ed8 at {username='slave'} at 127.3.139.72:51701: session id = f014c4ad4c1342219a1004181ac33ed8-9ad40762dc784dfa9f3507e025db32b8
I20240502 07:59:48.014410 20163 tablet_copy_source_session.cc:205] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:48.015631 23250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ad40762dc784dfa9f3507e025db32b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:48.017292 23250 tablet_copy_client.cc:783] T 9ad40762dc784dfa9f3507e025db32b8 P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:48.017427 23250 tablet_copy_client.cc:647] T 9ad40762dc784dfa9f3507e025db32b8 P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:48.018776 23250 tablet_copy_client.cc:523] T 9ad40762dc784dfa9f3507e025db32b8 P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:48.019775 23250 tablet_bootstrap.cc:492] T 9ad40762dc784dfa9f3507e025db32b8 P f014c4ad4c1342219a1004181ac33ed8: Bootstrap starting.
I20240502 07:59:48.022614 23237 tablet_bootstrap.cc:492] T 1453e833d0ad494583c44b5496088f6d P bfb1b3d596ef474faa229a835329cf20: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:48.022822 23237 tablet_bootstrap.cc:492] T 1453e833d0ad494583c44b5496088f6d P bfb1b3d596ef474faa229a835329cf20: Bootstrap complete.
I20240502 07:59:48.022964 23237 ts_tablet_manager.cc:1367] T 1453e833d0ad494583c44b5496088f6d P bfb1b3d596ef474faa229a835329cf20: Time spent bootstrapping tablet: real 0.020s user 0.016s sys 0.001s
I20240502 07:59:48.023412 23237 raft_consensus.cc:348] T 1453e833d0ad494583c44b5496088f6d P bfb1b3d596ef474faa229a835329cf20 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } }
I20240502 07:59:48.023550 23237 raft_consensus.cc:729] T 1453e833d0ad494583c44b5496088f6d P bfb1b3d596ef474faa229a835329cf20 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: bfb1b3d596ef474faa229a835329cf20, State: Initialized, Role: LEARNER
I20240502 07:59:48.023689 23237 consensus_queue.cc:260] T 1453e833d0ad494583c44b5496088f6d P bfb1b3d596ef474faa229a835329cf20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } }
I20240502 07:59:48.024026 23237 ts_tablet_manager.cc:1398] T 1453e833d0ad494583c44b5496088f6d P bfb1b3d596ef474faa229a835329cf20: Time spent starting tablet: real 0.001s user 0.005s sys 0.000s
I20240502 07:59:48.024576 20955 tablet_copy_service.cc:342] P b5f471f8b5944986a7d878cf6ac550b8: Request end of tablet copy session bfb1b3d596ef474faa229a835329cf20-1453e833d0ad494583c44b5496088f6d received from {username='slave'} at 127.3.139.73:43417
I20240502 07:59:48.024673 20955 tablet_copy_service.cc:433] P b5f471f8b5944986a7d878cf6ac550b8: ending tablet copy session bfb1b3d596ef474faa229a835329cf20-1453e833d0ad494583c44b5496088f6d on tablet 1453e833d0ad494583c44b5496088f6d with peer bfb1b3d596ef474faa229a835329cf20
I20240502 07:59:48.025954 23237 ts_tablet_manager.cc:902] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P bfb1b3d596ef474faa229a835329cf20: Initiating tablet copy from peer b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577)
I20240502 07:59:48.026335 23237 tablet_copy_client.cc:308] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P bfb1b3d596ef474faa229a835329cf20: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.71:39577
I20240502 07:59:48.026701 20955 tablet_copy_service.cc:140] P b5f471f8b5944986a7d878cf6ac550b8: Received BeginTabletCopySession request for tablet f588a3ec49a24b1a9a9e19cdf8c30ca5 from peer bfb1b3d596ef474faa229a835329cf20 ({username='slave'} at 127.3.139.73:43417)
I20240502 07:59:48.026782 20955 tablet_copy_service.cc:161] P b5f471f8b5944986a7d878cf6ac550b8: Beginning new tablet copy session on tablet f588a3ec49a24b1a9a9e19cdf8c30ca5 from peer bfb1b3d596ef474faa229a835329cf20 at {username='slave'} at 127.3.139.73:43417: session id = bfb1b3d596ef474faa229a835329cf20-f588a3ec49a24b1a9a9e19cdf8c30ca5
I20240502 07:59:48.027576 20955 tablet_copy_source_session.cc:205] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:48.028260 23237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f588a3ec49a24b1a9a9e19cdf8c30ca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:48.031605 23237 tablet_copy_client.cc:783] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P bfb1b3d596ef474faa229a835329cf20: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:48.031772 23237 tablet_copy_client.cc:647] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P bfb1b3d596ef474faa229a835329cf20: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:48.032786 23237 tablet_copy_client.cc:523] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P bfb1b3d596ef474faa229a835329cf20: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:48.033740 23237 tablet_bootstrap.cc:492] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P bfb1b3d596ef474faa229a835329cf20: Bootstrap starting.
I20240502 07:59:48.039750 23250 tablet_bootstrap.cc:492] T 9ad40762dc784dfa9f3507e025db32b8 P f014c4ad4c1342219a1004181ac33ed8: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:48.039978 23250 tablet_bootstrap.cc:492] T 9ad40762dc784dfa9f3507e025db32b8 P f014c4ad4c1342219a1004181ac33ed8: Bootstrap complete.
I20240502 07:59:48.040125 23250 ts_tablet_manager.cc:1367] T 9ad40762dc784dfa9f3507e025db32b8 P f014c4ad4c1342219a1004181ac33ed8: Time spent bootstrapping tablet: real 0.020s user 0.020s sys 0.000s
I20240502 07:59:48.040488 23250 raft_consensus.cc:348] T 9ad40762dc784dfa9f3507e025db32b8 P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } }
I20240502 07:59:48.040589 23250 raft_consensus.cc:729] T 9ad40762dc784dfa9f3507e025db32b8 P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f014c4ad4c1342219a1004181ac33ed8, State: Initialized, Role: LEARNER
I20240502 07:59:48.040668 23250 consensus_queue.cc:260] T 9ad40762dc784dfa9f3507e025db32b8 P f014c4ad4c1342219a1004181ac33ed8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } }
I20240502 07:59:48.040900 23250 ts_tablet_manager.cc:1398] T 9ad40762dc784dfa9f3507e025db32b8 P f014c4ad4c1342219a1004181ac33ed8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:48.041378 20163 tablet_copy_service.cc:342] P 29a7180fe4574c09a8dce7c27500aab2: Request end of tablet copy session f014c4ad4c1342219a1004181ac33ed8-9ad40762dc784dfa9f3507e025db32b8 received from {username='slave'} at 127.3.139.72:51701
I20240502 07:59:48.041468 20163 tablet_copy_service.cc:433] P 29a7180fe4574c09a8dce7c27500aab2: ending tablet copy session f014c4ad4c1342219a1004181ac33ed8-9ad40762dc784dfa9f3507e025db32b8 on tablet 9ad40762dc784dfa9f3507e025db32b8 with peer f014c4ad4c1342219a1004181ac33ed8
I20240502 07:59:48.051429 23237 tablet_bootstrap.cc:492] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P bfb1b3d596ef474faa229a835329cf20: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:48.051627 23237 tablet_bootstrap.cc:492] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P bfb1b3d596ef474faa229a835329cf20: Bootstrap complete.
I20240502 07:59:48.051761 23237 ts_tablet_manager.cc:1367] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P bfb1b3d596ef474faa229a835329cf20: Time spent bootstrapping tablet: real 0.018s user 0.018s sys 0.000s
I20240502 07:59:48.052081 23237 raft_consensus.cc:348] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P bfb1b3d596ef474faa229a835329cf20 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } }
I20240502 07:59:48.052187 23237 raft_consensus.cc:729] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P bfb1b3d596ef474faa229a835329cf20 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: bfb1b3d596ef474faa229a835329cf20, State: Initialized, Role: LEARNER
I20240502 07:59:48.052273 23237 consensus_queue.cc:260] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P bfb1b3d596ef474faa229a835329cf20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } }
I20240502 07:59:48.052487 23237 ts_tablet_manager.cc:1398] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P bfb1b3d596ef474faa229a835329cf20: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:48.052879 20955 tablet_copy_service.cc:342] P b5f471f8b5944986a7d878cf6ac550b8: Request end of tablet copy session bfb1b3d596ef474faa229a835329cf20-f588a3ec49a24b1a9a9e19cdf8c30ca5 received from {username='slave'} at 127.3.139.73:43417
I20240502 07:59:48.052954 20955 tablet_copy_service.cc:433] P b5f471f8b5944986a7d878cf6ac550b8: ending tablet copy session bfb1b3d596ef474faa229a835329cf20-f588a3ec49a24b1a9a9e19cdf8c30ca5 on tablet f588a3ec49a24b1a9a9e19cdf8c30ca5 with peer bfb1b3d596ef474faa229a835329cf20
I20240502 07:59:48.271569 22476 raft_consensus.cc:1203] T 0e2908de10684c84af542f16ff8727cb P bfb1b3d596ef474faa229a835329cf20 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240502 07:59:48.358125 22609 raft_consensus.cc:1203] T 6a77bda800d44c4bbe1b082a5ea05b55 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240502 07:59:48.362244 22741 raft_consensus.cc:1203] T 574fdd7a3c114ac9928ac12d1a7a4186 P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240502 07:59:48.367478 22741 raft_consensus.cc:1203] T 04659b5e3504411987dafc49a5fa5ea7 P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240502 07:59:48.421196 22476 raft_consensus.cc:1203] T 331368e52904494fa5ecf1ef594821fc P bfb1b3d596ef474faa229a835329cf20 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240502 07:59:48.431350 22476 raft_consensus.cc:1203] T 1453e833d0ad494583c44b5496088f6d P bfb1b3d596ef474faa229a835329cf20 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240502 07:59:48.462975 22476 raft_consensus.cc:1203] T e61fe3cb61c240f9b2c898aff1951144 P bfb1b3d596ef474faa229a835329cf20 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240502 07:59:48.491708 22476 raft_consensus.cc:1203] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P bfb1b3d596ef474faa229a835329cf20 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240502 07:59:48.512897 22741 raft_consensus.cc:1203] T 0f03a9b07a8c40bdbdf3ff714302a3ea P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240502 07:59:48.534446 22741 raft_consensus.cc:1203] T 9ad40762dc784dfa9f3507e025db32b8 P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240502 07:59:48.759196 23222 raft_consensus.cc:1050] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191: attempting to promote NON_VOTER bfb1b3d596ef474faa229a835329cf20 to VOTER
I20240502 07:59:48.759693 23222 consensus_queue.cc:237] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } }
I20240502 07:59:48.761490 20405 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:48.761632 20539 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:48.761775 20668 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:48.761855 23437 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:48.761937 20143 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:48.762219 22476 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P bfb1b3d596ef474faa229a835329cf20 [term 1 LEARNER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:48.762285 23438 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:48.762645 22993 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:48.762830 23439 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:48.763168 23436 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:48.763900 20802 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:48.764241 23439 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:48.765563 20933 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:48.765883 23439 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:48.766654 23438 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.767024 20405 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.767853 22476 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.768069 20933 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.769434 20143 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.770015 19992 catalog_manager.cc:5484] T 0e2908de10684c84af542f16ff8727cb P 35642aa2099c40e59bc2092604078e07 reported cstate change: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.771251 20539 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.771883 20802 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.772590 20668 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.778384 20273 consensus_queue.cc:237] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } }
I20240502 07:59:48.779809 22476 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:48.779846 20933 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:48.780133 20405 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:48.780215 20143 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:48.780184 23437 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:48.780313 23222 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:48.780490 20668 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:48.780496 23326 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:48.780860 23437 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:48.781054 23222 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:48.781668 20539 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:48.782029 23437 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:48.783071 22993 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.783444 20933 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.783721 20668 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.785212 19987 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 0e2908de10684c84af542f16ff8727cb with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:48.785617 20405 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.785629 20143 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.785859 22476 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.788058 20001 catalog_manager.cc:5484] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2 reported cstate change: config changed from index 6 to 7, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New cstate: current_term: 1 leader_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.788874 20539 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.790448 20781 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e2908de10684c84af542f16ff8727cb with delete_type TABLET_DATA_TOMBSTONED (TS c15e23a34059456788f9775031e27ec0 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:45388
I20240502 07:59:48.791656 23452 tablet_replica.cc:330] T 0e2908de10684c84af542f16ff8727cb P c15e23a34059456788f9775031e27ec0: stopping tablet replica
I20240502 07:59:48.791988 23452 raft_consensus.cc:2229] T 0e2908de10684c84af542f16ff8727cb P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:48.792313 23452 raft_consensus.cc:2258] T 0e2908de10684c84af542f16ff8727cb P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:48.793406 23452 ts_tablet_manager.cc:1875] T 0e2908de10684c84af542f16ff8727cb P c15e23a34059456788f9775031e27ec0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:48.795109 23452 ts_tablet_manager.cc:1888] T 0e2908de10684c84af542f16ff8727cb P c15e23a34059456788f9775031e27ec0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 07:59:48.795183 23452 log.cc:1196] T 0e2908de10684c84af542f16ff8727cb P c15e23a34059456788f9775031e27ec0: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/wal/wals/0e2908de10684c84af542f16ff8727cb
I20240502 07:59:48.795640 19986 catalog_manager.cc:4830] TS c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085): tablet 0e2908de10684c84af542f16ff8727cb (table rebalance_test_table_4 [id=8edc4db66dcd46ee8429441e0ce671a3]) successfully deleted
I20240502 07:59:48.807371 23012 raft_consensus.cc:1050] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0: attempting to promote NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 to VOTER
I20240502 07:59:48.807775 23012 consensus_queue.cc:237] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } }
I20240502 07:59:48.809518 20273 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:48.809518 20668 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:48.809585 20933 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:48.809835 20143 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:48.809831 23455 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:48.809947 20539 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:48.810292 20405 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:48.810314 23454 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:48.810482 23012 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:48.810508 22609 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:48.810662 23455 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:48.810845 23342 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:48.811028 23343 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:48.811225 23012 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:48.813829 23454 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:48.814132 20273 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:48.814360 20933 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:48.815639 20143 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:48.815738 20668 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:48.816692 19993 catalog_manager.cc:5484] T 6a77bda800d44c4bbe1b082a5ea05b55 P b5f471f8b5944986a7d878cf6ac550b8 reported cstate change: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c15e23a34059456788f9775031e27ec0" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:48.818107 20539 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:48.819415 20405 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:48.822481 22609 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:48.826355 20802 consensus_queue.cc:237] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } }
I20240502 07:59:48.827715 20405 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:48.828080 20668 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:48.828279 20273 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:48.828258 23012 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:48.828474 23455 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:48.828640 20143 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:48.828821 22609 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:48.829373 20933 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:48.829672 23456 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:48.829823 23012 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:48.830004 23343 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:48.830353 23456 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:48.831599 23343 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:48.832276 20143 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:48.832734 20933 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:48.833608 20405 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:48.833787 19986 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 6a77bda800d44c4bbe1b082a5ea05b55 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:48.833870 20668 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:48.834314 23349 raft_consensus.cc:1050] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07: attempting to promote NON_VOTER f014c4ad4c1342219a1004181ac33ed8 to VOTER
I20240502 07:59:48.834710 20273 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:48.834829 23349 consensus_queue.cc:237] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } }
I20240502 07:59:48.836531 19995 catalog_manager.cc:5484] T 6a77bda800d44c4bbe1b082a5ea05b55 P 35642aa2099c40e59bc2092604078e07 reported cstate change: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New cstate: current_term: 1 leader_uuid: "c15e23a34059456788f9775031e27ec0" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:48.837759 20142 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:48.838224 23355 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:48.838507 20537 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:48.838730 20517 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a77bda800d44c4bbe1b082a5ea05b55 with delete_type TABLET_DATA_TOMBSTONED (TS e951f009258e473cb57cf8773c7975d4 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:43532
I20240502 07:59:48.839202 23470 tablet_replica.cc:330] T 6a77bda800d44c4bbe1b082a5ea05b55 P e951f009258e473cb57cf8773c7975d4: stopping tablet replica
I20240502 07:59:48.839388 20933 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:48.839617 23470 raft_consensus.cc:2229] T 6a77bda800d44c4bbe1b082a5ea05b55 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:48.839589 23355 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:48.839900 23355 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:48.840824 23470 raft_consensus.cc:2258] T 6a77bda800d44c4bbe1b082a5ea05b55 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:48.841379 20802 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:48.841967 20275 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:48.842015 20668 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:48.842077 22741 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:48.842141 23355 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:48.842418 23355 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:48.842505 23456 raft_consensus.cc:1050] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0: attempting to promote NON_VOTER bfb1b3d596ef474faa229a835329cf20 to VOTER
I20240502 07:59:48.842953 23456 consensus_queue.cc:237] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } }
I20240502 07:59:48.844115 22609 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:48.844646 23355 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:48.845009 22963 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:48.845070 22476 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P bfb1b3d596ef474faa229a835329cf20 [term 1 LEARNER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:48.845438 23455 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:48.848338 23470 ts_tablet_manager.cc:1875] T 6a77bda800d44c4bbe1b082a5ea05b55 P e951f009258e473cb57cf8773c7975d4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:48.848555 23465 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:48.849690 20668 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:48.851438 20143 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:48.851725 20539 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:48.852685 19992 catalog_manager.cc:5484] T 574fdd7a3c114ac9928ac12d1a7a4186 P b443e68b6212450286869124684bdfff reported cstate change: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "35642aa2099c40e59bc2092604078e07" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:48.853660 23470 ts_tablet_manager.cc:1888] T 6a77bda800d44c4bbe1b082a5ea05b55 P e951f009258e473cb57cf8773c7975d4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 07:59:48.853724 20933 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:48.854772 23470 log.cc:1196] T 6a77bda800d44c4bbe1b082a5ea05b55 P e951f009258e473cb57cf8773c7975d4: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal/wals/6a77bda800d44c4bbe1b082a5ea05b55
I20240502 07:59:48.854954 20935 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:48.855430 20539 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:48.855818 23455 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:48.856416 19986 catalog_manager.cc:4830] TS e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315): tablet 6a77bda800d44c4bbe1b082a5ea05b55 (table rebalance_test_table_5 [id=ed17eb489c6e490e9427e387589e51fe]) successfully deleted
I20240502 07:59:48.857800 23454 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:48.857687 20273 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:48.857930 22741 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:48.859373 20273 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:48.859750 23454 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:48.860949 20143 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:48.860997 20405 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:48.861408 23012 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:48.861709 20668 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:48.861716 23012 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:48.862093 23012 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:48.864060 23454 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.864431 20273 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.866648 20143 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.866971 20539 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.868115 23465 raft_consensus.cc:1050] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07: attempting to promote NON_VOTER bfb1b3d596ef474faa229a835329cf20 to VOTER
I20240502 07:59:48.868563 23465 consensus_queue.cc:237] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } }
I20240502 07:59:48.869289 20001 catalog_manager.cc:5484] T 331368e52904494fa5ecf1ef594821fc P 29a7180fe4574c09a8dce7c27500aab2 reported cstate change: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c15e23a34059456788f9775031e27ec0" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.869834 22476 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.870316 22475 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P bfb1b3d596ef474faa229a835329cf20 [term 1 LEARNER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:48.870738 23467 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:48.870977 20143 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:48.872323 23467 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:48.873551 20539 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:48.873914 20935 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:48.874172 20273 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:48.874255 23355 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:48.874395 23466 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:48.875555 20671 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:48.875664 23355 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:48.876895 20802 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:48.877403 23355 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:48.878419 20800 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:48.878821 23355 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:48.878999 23465 raft_consensus.cc:1050] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07: attempting to promote NON_VOTER f014c4ad4c1342219a1004181ac33ed8 to VOTER
I20240502 07:59:48.880098 23465 consensus_queue.cc:237] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } }
I20240502 07:59:48.880390 20933 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.882242 20933 raft_consensus.cc:1261] T 04659b5e3504411987dafc49a5fa5ea7 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:48.882474 20142 raft_consensus.cc:1261] T 04659b5e3504411987dafc49a5fa5ea7 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:48.882514 20539 raft_consensus.cc:1261] T 04659b5e3504411987dafc49a5fa5ea7 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:48.882572 23466 consensus_queue.cc:1035] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:48.883126 23349 consensus_queue.cc:1035] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:48.883451 23349 consensus_queue.cc:1035] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:48.884989 20275 raft_consensus.cc:1261] T 04659b5e3504411987dafc49a5fa5ea7 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:48.885376 22741 raft_consensus.cc:1261] T 04659b5e3504411987dafc49a5fa5ea7 P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:48.885522 20406 consensus_queue.cc:237] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } }
I20240502 07:59:48.885846 20671 raft_consensus.cc:1261] T 04659b5e3504411987dafc49a5fa5ea7 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:48.885847 23466 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.887408 20802 raft_consensus.cc:1261] T 04659b5e3504411987dafc49a5fa5ea7 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:48.888094 23355 consensus_queue.cc:1035] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:48.888206 20140 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:48.888789 23355 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:48.888723 20935 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.888882 20933 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:48.889242 23355 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:48.890313 20802 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.890487 20275 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:48.891369 23355 consensus_queue.cc:1035] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:48.891688 23355 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:48.891871 20671 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:48.891918 23355 consensus_queue.cc:1035] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:48.892458 19999 catalog_manager.cc:5484] T e61fe3cb61c240f9b2c898aff1951144 P b5f471f8b5944986a7d878cf6ac550b8 reported cstate change: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "35642aa2099c40e59bc2092604078e07" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.893258 22476 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.893474 20800 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:48.893510 20539 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.895179 23469 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:48.896821 20143 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.898422 23466 raft_consensus.cc:2932] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:48.898854 20143 raft_consensus.cc:2932] T 04659b5e3504411987dafc49a5fa5ea7 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:48.900159 20935 raft_consensus.cc:2932] T 04659b5e3504411987dafc49a5fa5ea7 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:48.901917 20670 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.902475 22740 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:48.902464 19993 catalog_manager.cc:5484] T 04659b5e3504411987dafc49a5fa5ea7 P b5f471f8b5944986a7d878cf6ac550b8 reported cstate change: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "35642aa2099c40e59bc2092604078e07" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:48.904023 20802 raft_consensus.cc:2932] T 04659b5e3504411987dafc49a5fa5ea7 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:48.904196 23465 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:48.905041 23469 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:48.906051 20539 raft_consensus.cc:2932] T 04659b5e3504411987dafc49a5fa5ea7 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:48.906507 20671 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:48.906708 20670 raft_consensus.cc:2932] T 04659b5e3504411987dafc49a5fa5ea7 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:48.907199 23465 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:48.907842 20933 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:48.908869 19988 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 574fdd7a3c114ac9928ac12d1a7a4186 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:48.909914 20802 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:48.910141 19992 catalog_manager.cc:5484] T 574fdd7a3c114ac9928ac12d1a7a4186 P b443e68b6212450286869124684bdfff reported cstate change: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New cstate: current_term: 1 leader_uuid: "35642aa2099c40e59bc2092604078e07" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:48.911520 20142 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:48.911522 20275 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:48.912356 22740 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:48.915597 20517 tablet_service.cc:1514] Processing DeleteTablet for tablet 574fdd7a3c114ac9928ac12d1a7a4186 with delete_type TABLET_DATA_TOMBSTONED (TS e951f009258e473cb57cf8773c7975d4 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:43532
I20240502 07:59:48.915840 23470 tablet_replica.cc:330] T 574fdd7a3c114ac9928ac12d1a7a4186 P e951f009258e473cb57cf8773c7975d4: stopping tablet replica
I20240502 07:59:48.915956 23470 raft_consensus.cc:2229] T 574fdd7a3c114ac9928ac12d1a7a4186 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:48.916136 23470 raft_consensus.cc:2258] T 574fdd7a3c114ac9928ac12d1a7a4186 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:48.916803 22741 raft_consensus.cc:2932] T 04659b5e3504411987dafc49a5fa5ea7 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:48.917167 23470 ts_tablet_manager.cc:1875] T 574fdd7a3c114ac9928ac12d1a7a4186 P e951f009258e473cb57cf8773c7975d4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:48.917781 23505 consensus_queue.cc:1035] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:48.918828 20275 raft_consensus.cc:2932] T 04659b5e3504411987dafc49a5fa5ea7 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:48.919672 20405 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.921342 23470 ts_tablet_manager.cc:1888] T 574fdd7a3c114ac9928ac12d1a7a4186 P e951f009258e473cb57cf8773c7975d4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 07:59:48.921411 23470 log.cc:1196] T 574fdd7a3c114ac9928ac12d1a7a4186 P e951f009258e473cb57cf8773c7975d4: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal/wals/574fdd7a3c114ac9928ac12d1a7a4186
I20240502 07:59:48.921715 20273 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.922663 20668 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.922860 19986 catalog_manager.cc:4830] TS e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315): tablet 574fdd7a3c114ac9928ac12d1a7a4186 (table rebalance_test_table_6 [id=db1e20e82b894565a302e53bd7791607]) successfully deleted
I20240502 07:59:48.926203 20405 consensus_queue.cc:237] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } }
I20240502 07:59:48.927580 20800 consensus_queue.cc:237] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } }
I20240502 07:59:48.928231 20933 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:48.928566 23465 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:48.928741 20275 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:48.929399 20143 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:48.929430 20935 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:48.929704 22475 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:48.929762 23454 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:48.929913 23012 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:48.929704 22476 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:48.930194 23349 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:48.930979 23454 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:48.931193 20406 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:48.931301 23349 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:48.931659 20275 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:48.932041 23012 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:48.932291 23343 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:48.933084 20537 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:48.933137 20539 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:48.933137 20142 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:48.933581 20800 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:48.933692 23342 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:48.933888 23012 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.934288 20273 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.934726 22475 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.936457 20001 catalog_manager.cc:5484] T 331368e52904494fa5ecf1ef594821fc P b7604c0eb3bf4a7cb260b564a94a1191 reported cstate change: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New cstate: current_term: 1 leader_uuid: "c15e23a34059456788f9775031e27ec0" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.936489 20933 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.937158 19986 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 331368e52904494fa5ecf1ef594821fc with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:48.937258 23468 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:48.937566 23468 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:48.937863 23468 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:48.938643 20142 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.940220 20649 tablet_service.cc:1514] Processing DeleteTablet for tablet 331368e52904494fa5ecf1ef594821fc with delete_type TABLET_DATA_TOMBSTONED (TS b443e68b6212450286869124684bdfff not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:53090
I20240502 07:59:48.941586 23520 tablet_replica.cc:330] T 331368e52904494fa5ecf1ef594821fc P b443e68b6212450286869124684bdfff: stopping tablet replica
I20240502 07:59:48.941740 23468 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.941931 23520 raft_consensus.cc:2229] T 331368e52904494fa5ecf1ef594821fc P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:48.942137 23520 raft_consensus.cc:2258] T 331368e52904494fa5ecf1ef594821fc P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:48.943233 19988 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet e61fe3cb61c240f9b2c898aff1951144 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:48.943573 23520 ts_tablet_manager.cc:1875] T 331368e52904494fa5ecf1ef594821fc P b443e68b6212450286869124684bdfff: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:48.943836 20933 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.944661 22475 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.945950 23520 ts_tablet_manager.cc:1888] T 331368e52904494fa5ecf1ef594821fc P b443e68b6212450286869124684bdfff: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 07:59:48.946035 23520 log.cc:1196] T 331368e52904494fa5ecf1ef594821fc P b443e68b6212450286869124684bdfff: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/wal/wals/331368e52904494fa5ecf1ef594821fc
I20240502 07:59:48.946079 19995 catalog_manager.cc:5484] T e61fe3cb61c240f9b2c898aff1951144 P b5f471f8b5944986a7d878cf6ac550b8 reported cstate change: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New cstate: current_term: 1 leader_uuid: "35642aa2099c40e59bc2092604078e07" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.946765 19987 catalog_manager.cc:4830] TS b443e68b6212450286869124684bdfff (127.3.139.69:37923): tablet 331368e52904494fa5ecf1ef594821fc (table rebalance_test_table_4 [id=8edc4db66dcd46ee8429441e0ce671a3]) successfully deleted
I20240502 07:59:48.947130 20539 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.947284 20143 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.947314 20537 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.948870 20649 tablet_service.cc:1514] Processing DeleteTablet for tablet e61fe3cb61c240f9b2c898aff1951144 with delete_type TABLET_DATA_TOMBSTONED (TS b443e68b6212450286869124684bdfff not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:53090
I20240502 07:59:48.949472 23520 tablet_replica.cc:330] T e61fe3cb61c240f9b2c898aff1951144 P b443e68b6212450286869124684bdfff: stopping tablet replica
I20240502 07:59:48.949625 23520 raft_consensus.cc:2229] T e61fe3cb61c240f9b2c898aff1951144 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:48.949826 23520 raft_consensus.cc:2258] T e61fe3cb61c240f9b2c898aff1951144 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:48.950192 20405 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.950683 23520 ts_tablet_manager.cc:1875] T e61fe3cb61c240f9b2c898aff1951144 P b443e68b6212450286869124684bdfff: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:48.952140 20802 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:48.952721 23520 ts_tablet_manager.cc:1888] T e61fe3cb61c240f9b2c898aff1951144 P b443e68b6212450286869124684bdfff: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 07:59:48.952821 23520 log.cc:1196] T e61fe3cb61c240f9b2c898aff1951144 P b443e68b6212450286869124684bdfff: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/wal/wals/e61fe3cb61c240f9b2c898aff1951144
I20240502 07:59:48.953368 19987 catalog_manager.cc:4830] TS b443e68b6212450286869124684bdfff (127.3.139.69:37923): tablet e61fe3cb61c240f9b2c898aff1951144 (table rebalance_test_table_3 [id=1388635dd26f4d3791ee92dfbf8d374a]) successfully deleted
I20240502 07:59:48.954648 20275 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:49.051769 23383 raft_consensus.cc:1050] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8: attempting to promote NON_VOTER bfb1b3d596ef474faa229a835329cf20 to VOTER
I20240502 07:59:49.052338 23383 consensus_queue.cc:237] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } }
I20240502 07:59:49.053987 22475 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P bfb1b3d596ef474faa229a835329cf20 [term 1 LEARNER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:49.054227 20802 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:49.054390 23382 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:49.054489 20143 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:49.054474 20539 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:49.054658 23559 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:49.054975 20668 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:49.055155 20275 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:49.055518 20405 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:49.056077 23382 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:49.056253 23561 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:49.056485 23560 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:49.056687 23562 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:49.056809 23559 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:49.059381 23560 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:49.060257 20405 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:49.060298 20668 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:49.060701 20143 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:49.061626 22475 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:49.062003 20275 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:49.063371 20001 catalog_manager.cc:5484] T 1453e833d0ad494583c44b5496088f6d P b443e68b6212450286869124684bdfff reported cstate change: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b5f471f8b5944986a7d878cf6ac550b8" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:49.063696 20539 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:49.066444 20802 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
W20240502 07:59:49.068926 23382 consensus_queue.cc:1159] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "c15e23a34059456788f9775031e27ec0" responder_term: 1 status { last_received { term: 1 index: 6 } last_committed_idx: 6 last_received_current_leader { term: 1 index: 6 } }
I20240502 07:59:49.069105 20933 consensus_queue.cc:237] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } }
I20240502 07:59:49.070432 22475 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:49.070542 20668 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:49.070863 23561 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:49.070919 20539 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:49.071022 20143 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:49.071071 23382 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:49.071280 23557 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:49.071441 20275 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:49.071455 23382 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:49.071719 23559 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:49.071759 20405 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:49.072242 23561 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:49.073798 23562 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:49.074158 20143 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:49.075774 19988 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 1453e833d0ad494583c44b5496088f6d with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:49.075974 20539 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:49.076611 19993 catalog_manager.cc:5484] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 reported cstate change: config changed from index 6 to 7, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New cstate: current_term: 1 leader_uuid: "b5f471f8b5944986a7d878cf6ac550b8" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:49.077019 20275 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:49.078768 22475 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:49.079388 20405 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:49.079766 20668 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:49.081339 20781 tablet_service.cc:1514] Processing DeleteTablet for tablet 1453e833d0ad494583c44b5496088f6d with delete_type TABLET_DATA_TOMBSTONED (TS c15e23a34059456788f9775031e27ec0 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:45388
I20240502 07:59:49.081568 23452 tablet_replica.cc:330] T 1453e833d0ad494583c44b5496088f6d P c15e23a34059456788f9775031e27ec0: stopping tablet replica
I20240502 07:59:49.081708 23452 raft_consensus.cc:2229] T 1453e833d0ad494583c44b5496088f6d P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:49.081887 23452 raft_consensus.cc:2258] T 1453e833d0ad494583c44b5496088f6d P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:49.082616 23452 ts_tablet_manager.cc:1875] T 1453e833d0ad494583c44b5496088f6d P c15e23a34059456788f9775031e27ec0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:49.084533 23452 ts_tablet_manager.cc:1888] T 1453e833d0ad494583c44b5496088f6d P c15e23a34059456788f9775031e27ec0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 07:59:49.084610 23452 log.cc:1196] T 1453e833d0ad494583c44b5496088f6d P c15e23a34059456788f9775031e27ec0: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/wal/wals/1453e833d0ad494583c44b5496088f6d
I20240502 07:59:49.085017 19986 catalog_manager.cc:4830] TS c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085): tablet 1453e833d0ad494583c44b5496088f6d (table rebalance_test_table_1 [id=dd634821c68c405c9e104a7053619a61]) successfully deleted
I20240502 07:59:49.094671 23274 raft_consensus.cc:1050] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff: attempting to promote NON_VOTER f014c4ad4c1342219a1004181ac33ed8 to VOTER
I20240502 07:59:49.095081 23274 consensus_queue.cc:237] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } }
I20240502 07:59:49.096530 20405 raft_consensus.cc:1261] T 0f03a9b07a8c40bdbdf3ff714302a3ea P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:49.096774 20539 raft_consensus.cc:1261] T 0f03a9b07a8c40bdbdf3ff714302a3ea P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:49.096827 20802 raft_consensus.cc:1261] T 0f03a9b07a8c40bdbdf3ff714302a3ea P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:49.097142 20143 raft_consensus.cc:1261] T 0f03a9b07a8c40bdbdf3ff714302a3ea P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:49.097143 23571 consensus_queue.cc:1035] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:49.097257 20275 raft_consensus.cc:1261] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:49.097359 23574 consensus_queue.cc:1035] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:49.097589 23274 consensus_queue.cc:1035] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:49.097692 22741 raft_consensus.cc:1261] T 0f03a9b07a8c40bdbdf3ff714302a3ea P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:49.097898 23274 consensus_queue.cc:1035] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:49.098071 23099 consensus_queue.cc:1035] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:49.098109 20933 raft_consensus.cc:1261] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:49.098377 23099 consensus_queue.cc:1035] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:49.098510 23572 consensus_queue.cc:1035] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:49.101537 23274 raft_consensus.cc:2932] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:49.101802 20802 raft_consensus.cc:2932] T 0f03a9b07a8c40bdbdf3ff714302a3ea P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:49.102322 20537 raft_consensus.cc:2932] T 0f03a9b07a8c40bdbdf3ff714302a3ea P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:49.103847 22741 raft_consensus.cc:2932] T 0f03a9b07a8c40bdbdf3ff714302a3ea P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:49.104005 20933 raft_consensus.cc:2932] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:49.104768 20275 raft_consensus.cc:2932] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:49.105847 19993 catalog_manager.cc:5484] T 0f03a9b07a8c40bdbdf3ff714302a3ea P c15e23a34059456788f9775031e27ec0 reported cstate change: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b443e68b6212450286869124684bdfff" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:49.105857 20143 raft_consensus.cc:2932] T 0f03a9b07a8c40bdbdf3ff714302a3ea P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:49.107450 23557 raft_consensus.cc:1050] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8: attempting to promote NON_VOTER bfb1b3d596ef474faa229a835329cf20 to VOTER
I20240502 07:59:49.107892 23557 consensus_queue.cc:237] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } }
I20240502 07:59:49.109496 22475 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P bfb1b3d596ef474faa229a835329cf20 [term 1 LEARNER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:49.109637 20802 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:49.110081 23560 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:49.110157 20143 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:49.110296 20668 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:49.110509 20275 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:49.110848 23383 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:49.111166 23383 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:49.111450 23383 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:49.111518 20406 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:49.112017 23382 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:49.113611 20537 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:49.113988 23560 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:49.115970 23560 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:49.116251 23382 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:49.116357 22475 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:49.117303 20406 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:49.117866 20802 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:49.117887 20668 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:49.119203 20143 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:49.119874 20537 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:49.121162 19994 catalog_manager.cc:5484] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07 reported cstate change: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b5f471f8b5944986a7d878cf6ac550b8" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:49.125535 20405 raft_consensus.cc:2932] T 0f03a9b07a8c40bdbdf3ff714302a3ea P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:49.125756 20275 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:49.129544 20668 consensus_queue.cc:237] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } }
I20240502 07:59:49.130949 22741 raft_consensus.cc:1261] T 0f03a9b07a8c40bdbdf3ff714302a3ea P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:49.130978 20143 raft_consensus.cc:1261] T 0f03a9b07a8c40bdbdf3ff714302a3ea P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:49.131335 23274 consensus_queue.cc:1035] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:49.131453 20933 raft_consensus.cc:1261] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:49.131479 23571 consensus_queue.cc:1035] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:49.131546 20802 raft_consensus.cc:1261] T 0f03a9b07a8c40bdbdf3ff714302a3ea P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:49.131764 23573 consensus_queue.cc:1035] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:49.132066 23274 consensus_queue.cc:1035] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:49.132102 20405 raft_consensus.cc:1261] T 0f03a9b07a8c40bdbdf3ff714302a3ea P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:49.132519 20275 raft_consensus.cc:1261] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:49.133154 23571 consensus_queue.cc:1035] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:49.133452 23571 consensus_queue.cc:1035] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:49.134759 23572 raft_consensus.cc:2932] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:49.135200 22741 raft_consensus.cc:2932] T 0f03a9b07a8c40bdbdf3ff714302a3ea P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:49.135569 20405 raft_consensus.cc:2932] T 0f03a9b07a8c40bdbdf3ff714302a3ea P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:49.136129 19987 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 0f03a9b07a8c40bdbdf3ff714302a3ea with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:49.137177 19993 catalog_manager.cc:5484] T 0f03a9b07a8c40bdbdf3ff714302a3ea P f014c4ad4c1342219a1004181ac33ed8 reported cstate change: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New cstate: current_term: 1 leader_uuid: "b443e68b6212450286869124684bdfff" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:49.137331 20802 raft_consensus.cc:2932] T 0f03a9b07a8c40bdbdf3ff714302a3ea P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:49.137964 20143 raft_consensus.cc:2932] T 0f03a9b07a8c40bdbdf3ff714302a3ea P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:49.139549 20517 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f03a9b07a8c40bdbdf3ff714302a3ea with delete_type TABLET_DATA_TOMBSTONED (TS e951f009258e473cb57cf8773c7975d4 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:43532
I20240502 07:59:49.140069 23470 tablet_replica.cc:330] T 0f03a9b07a8c40bdbdf3ff714302a3ea P e951f009258e473cb57cf8773c7975d4: stopping tablet replica
I20240502 07:59:49.140206 23470 raft_consensus.cc:2229] T 0f03a9b07a8c40bdbdf3ff714302a3ea P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:49.140359 23470 raft_consensus.cc:2258] T 0f03a9b07a8c40bdbdf3ff714302a3ea P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:49.141259 20933 raft_consensus.cc:2932] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:49.142307 20272 raft_consensus.cc:2932] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:49.142524 23470 ts_tablet_manager.cc:1875] T 0f03a9b07a8c40bdbdf3ff714302a3ea P e951f009258e473cb57cf8773c7975d4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:49.144611 23470 ts_tablet_manager.cc:1888] T 0f03a9b07a8c40bdbdf3ff714302a3ea P e951f009258e473cb57cf8773c7975d4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 07:59:49.144675 23470 log.cc:1196] T 0f03a9b07a8c40bdbdf3ff714302a3ea P e951f009258e473cb57cf8773c7975d4: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal/wals/0f03a9b07a8c40bdbdf3ff714302a3ea
I20240502 07:59:49.148042 19986 catalog_manager.cc:4830] TS e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315): tablet 0f03a9b07a8c40bdbdf3ff714302a3ea (table rebalance_test_table_5 [id=ed17eb489c6e490e9427e387589e51fe]) successfully deleted
I20240502 07:59:49.149838 23075 raft_consensus.cc:1050] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2: attempting to promote NON_VOTER f014c4ad4c1342219a1004181ac33ed8 to VOTER
I20240502 07:59:49.150290 23075 consensus_queue.cc:237] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } }
I20240502 07:59:49.152076 22741 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:49.152264 20537 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:49.152372 20668 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:49.152614 23075 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:49.152654 20933 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:49.153203 23591 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:49.153414 23075 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:49.153609 23384 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:49.153971 20802 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:49.154309 23384 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:49.155009 20272 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:49.155438 23591 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:49.155620 20405 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:49.156064 23075 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:49.157764 23384 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:49.158059 20272 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:49.158138 20405 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:49.159586 22741 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:49.160702 20001 catalog_manager.cc:5484] T 9ad40762dc784dfa9f3507e025db32b8 P b7604c0eb3bf4a7cb260b564a94a1191 reported cstate change: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "29a7180fe4574c09a8dce7c27500aab2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:49.161898 20668 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:49.162765 20802 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:49.165885 20933 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:49.166380 20537 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:49.168838 20143 consensus_queue.cc:237] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } }
I20240502 07:59:49.170483 20668 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:49.170754 20933 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:49.170811 23592 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:49.170886 20272 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:49.171141 23075 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:49.171325 22741 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:49.171674 23075 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:49.171721 20537 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:49.171932 23075 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:49.172197 23075 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:49.173204 20405 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:49.173514 23591 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:49.174782 23591 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:49.175660 20405 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:49.176124 19986 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 9ad40762dc784dfa9f3507e025db32b8 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:49.176192 20537 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:49.176625 20668 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:49.176826 22741 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:49.178231 20001 catalog_manager.cc:5484] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 reported cstate change: config changed from index 6 to 7, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New cstate: current_term: 1 leader_uuid: "29a7180fe4574c09a8dce7c27500aab2" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:49.178606 20933 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:49.180899 20781 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ad40762dc784dfa9f3507e025db32b8 with delete_type TABLET_DATA_TOMBSTONED (TS c15e23a34059456788f9775031e27ec0 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:45388
I20240502 07:59:49.181088 23452 tablet_replica.cc:330] T 9ad40762dc784dfa9f3507e025db32b8 P c15e23a34059456788f9775031e27ec0: stopping tablet replica
I20240502 07:59:49.181210 23452 raft_consensus.cc:2229] T 9ad40762dc784dfa9f3507e025db32b8 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:49.181358 23452 raft_consensus.cc:2258] T 9ad40762dc784dfa9f3507e025db32b8 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:49.182211 20272 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:49.183796 23452 ts_tablet_manager.cc:1875] T 9ad40762dc784dfa9f3507e025db32b8 P c15e23a34059456788f9775031e27ec0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:49.185842 23452 ts_tablet_manager.cc:1888] T 9ad40762dc784dfa9f3507e025db32b8 P c15e23a34059456788f9775031e27ec0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 07:59:49.185900 23452 log.cc:1196] T 9ad40762dc784dfa9f3507e025db32b8 P c15e23a34059456788f9775031e27ec0: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/wal/wals/9ad40762dc784dfa9f3507e025db32b8
I20240502 07:59:49.186241 19986 catalog_manager.cc:4830] TS c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085): tablet 9ad40762dc784dfa9f3507e025db32b8 (table rebalance_test_table_6 [id=db1e20e82b894565a302e53bd7791607]) successfully deleted
I20240502 07:59:49.199165 20802 consensus_queue.cc:237] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:49.200639 20272 raft_consensus.cc:1261] T b73e0ce533014420af513429620af68c P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:49.200731 20933 raft_consensus.cc:1261] T b73e0ce533014420af513429620af68c P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:49.200757 20143 raft_consensus.cc:1261] T b73e0ce533014420af513429620af68c P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:49.201352 20405 raft_consensus.cc:1261] T b73e0ce533014420af513429620af68c P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:49.201517 20668 raft_consensus.cc:1261] T b73e0ce533014420af513429620af68c P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:49.201582 23456 consensus_queue.cc:1035] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.201771 20537 raft_consensus.cc:1261] T b73e0ce533014420af513429620af68c P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:49.201874 23455 consensus_queue.cc:1035] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.202031 23456 consensus_queue.cc:1035] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.202484 23455 consensus_queue.cc:1035] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.202627 23012 consensus_queue.cc:1035] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.203102 23012 consensus_queue.cc:1035] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.206161 23455 raft_consensus.cc:2932] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:49.206544 20933 raft_consensus.cc:2932] T b73e0ce533014420af513429620af68c P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:49.208035 20668 raft_consensus.cc:2932] T b73e0ce533014420af513429620af68c P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:49.209324 20537 raft_consensus.cc:2932] T b73e0ce533014420af513429620af68c P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:49.209822 19999 catalog_manager.cc:5484] T b73e0ce533014420af513429620af68c P b5f471f8b5944986a7d878cf6ac550b8 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "c15e23a34059456788f9775031e27ec0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:49.210152 20143 raft_consensus.cc:2932] T b73e0ce533014420af513429620af68c P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:49.212126 20405 raft_consensus.cc:2932] T b73e0ce533014420af513429620af68c P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:49.215066 20272 raft_consensus.cc:2932] T b73e0ce533014420af513429620af68c P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:49.219430 20802 consensus_queue.cc:237] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } }
I20240502 07:59:49.220734 20537 consensus_queue.cc:237] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:49.221437 20668 raft_consensus.cc:1261] T b73e0ce533014420af513429620af68c P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20240502 07:59:49.221628 20738 consensus_peers.cc:485] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b73e0ce533014420af513429620af68c. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:49.221880 20272 raft_consensus.cc:1261] T b73e0ce533014420af513429620af68c P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:49.221866 23455 consensus_queue.cc:1035] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:49.222373 20143 raft_consensus.cc:1261] T b73e0ce533014420af513429620af68c P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:49.222604 23455 consensus_queue.cc:1035] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:49.222743 23621 consensus_queue.cc:1035] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:49.223328 20405 raft_consensus.cc:1261] T b73e0ce533014420af513429620af68c P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:49.223371 20933 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:49.223637 20406 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:49.224279 20143 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:49.224296 20275 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:49.224452 20933 raft_consensus.cc:1261] T b73e0ce533014420af513429620af68c P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:49.224543 23623 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.224731 23509 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.224835 23343 consensus_queue.cc:1035] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:49.224862 22888 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.224988 23624 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.225908 20668 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:49.225901 20802 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:49.225908 23621 consensus_queue.cc:1035] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:49.226428 23626 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.226543 20537 raft_consensus.cc:1261] T b73e0ce533014420af513429620af68c P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:49.226891 22888 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.226919 23455 consensus_queue.cc:1035] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:49.227190 23343 raft_consensus.cc:2932] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:49.227892 20933 raft_consensus.cc:2932] T b73e0ce533014420af513429620af68c P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:49.228117 20143 raft_consensus.cc:2932] T b73e0ce533014420af513429620af68c P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:49.229813 19986 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet b73e0ce533014420af513429620af68c with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240502 07:59:49.229816 20405 raft_consensus.cc:2932] T b73e0ce533014420af513429620af68c P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:49.230274 19993 catalog_manager.cc:5484] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 reported cstate change: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New cstate: current_term: 1 leader_uuid: "c15e23a34059456788f9775031e27ec0" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:49.230576 20275 raft_consensus.cc:2932] T b73e0ce533014420af513429620af68c P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:49.232170 20671 raft_consensus.cc:2932] T b73e0ce533014420af513429620af68c P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:49.235240 23623 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:49.236404 20406 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:49.237272 20142 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:49.238356 20001 catalog_manager.cc:5484] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "e951f009258e473cb57cf8773c7975d4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: HEALTHY } } }
I20240502 07:59:49.239082 20668 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:49.240303 20802 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:49.241199 20933 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:49.243028 20272 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:49.244787 20539 consensus_queue.cc:237] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } }
I20240502 07:59:49.246894 20406 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:49.246783 20537 raft_consensus.cc:2932] T b73e0ce533014420af513429620af68c P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:49.246968 20933 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:49.247346 23625 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:49.247627 23625 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:49.248979 20142 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:49.249454 20272 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:49.249670 20668 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:49.249713 23625 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:49.249833 23509 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:49.250651 23625 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:49.251135 20800 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:49.251953 23625 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:49.252242 23633 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:49.252771 20142 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:49.253455 20668 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:49.253839 20272 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:49.255690 19986 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet a973de06330c487ba54a8a36b1e31b54 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240502 07:59:49.256635 20802 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:49.257522 19994 catalog_manager.cc:5484] T a973de06330c487ba54a8a36b1e31b54 P 29a7180fe4574c09a8dce7c27500aab2 reported cstate change: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New cstate: current_term: 1 leader_uuid: "e951f009258e473cb57cf8773c7975d4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:49.257547 20406 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:49.257755 20933 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
W20240502 07:59:49.263434 20474 consensus_peers.cc:485] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a973de06330c487ba54a8a36b1e31b54. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:49.268934 20668 consensus_queue.cc:237] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:49.270434 20272 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:49.270459 20935 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:49.270488 20406 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:49.270514 20142 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:49.270870 23574 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.271139 23099 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.271265 23574 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.271400 23573 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.271589 20802 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:49.271894 23099 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.271981 20537 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:49.273020 23099 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.273711 23574 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:49.274113 20406 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:49.274544 20935 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:49.276010 20802 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:49.276440 20001 catalog_manager.cc:5484] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "b443e68b6212450286869124684bdfff" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: HEALTHY } } }
I20240502 07:59:49.276625 20537 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:49.278949 20272 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:49.279914 20142 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:49.280045 20668 consensus_queue.cc:237] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } }
I20240502 07:59:49.281832 20142 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:49.282156 23274 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:49.282202 20935 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:49.282578 23274 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:49.282692 20537 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:49.283005 23274 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:49.283124 20802 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:49.283607 23274 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:49.284827 20406 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:49.285519 23574 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:49.287066 23574 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:49.287477 20406 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:49.287735 20802 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:49.288039 20272 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:49.288599 23571 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:49.288920 19987 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet bbae35bae8c047a6982e128ffb77ca86 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240502 07:59:49.290256 20001 catalog_manager.cc:5484] T bbae35bae8c047a6982e128ffb77ca86 P 35642aa2099c40e59bc2092604078e07 reported cstate change: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New cstate: current_term: 1 leader_uuid: "b443e68b6212450286869124684bdfff" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:49.290892 20272 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:49.291303 20537 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:49.291823 20935 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:49.294220 20142 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:49.295310 20935 consensus_queue.cc:237] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:49.297511 20802 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:49.297582 20668 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:49.297984 23560 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.298242 23383 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.298739 20142 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:49.299132 20406 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:49.299129 23557 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.299470 23557 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.299738 20272 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:49.300118 23561 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.300822 20537 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 07:59:49.301090 20606 consensus_peers.cc:485] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bbae35bae8c047a6982e128ffb77ca86. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:49.301440 23561 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.301954 23557 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:49.303447 20668 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:49.303970 20272 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:49.304734 20406 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:49.304850 20142 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:49.306439 20001 catalog_manager.cc:5484] T 15bb9480e88b41219cc91168cc438c70 P b7604c0eb3bf4a7cb260b564a94a1191 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "b5f471f8b5944986a7d878cf6ac550b8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:49.306663 20802 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:49.309010 20537 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:49.313750 20935 consensus_queue.cc:237] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } }
I20240502 07:59:49.315876 20406 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:49.316011 20668 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:49.316120 20802 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:49.316313 23557 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:49.316507 23560 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:49.316577 20142 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:49.316615 20537 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:49.316865 20272 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:49.317122 23561 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:49.317418 23561 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:49.317711 23561 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:49.319195 23557 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:49.319495 23382 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:49.319576 20272 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:49.320185 20142 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:49.321184 19988 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 15bb9480e88b41219cc91168cc438c70 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240502 07:59:49.321606 19999 catalog_manager.cc:5484] T 15bb9480e88b41219cc91168cc438c70 P b7604c0eb3bf4a7cb260b564a94a1191 reported cstate change: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New cstate: current_term: 1 leader_uuid: "b5f471f8b5944986a7d878cf6ac550b8" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:49.321805 20668 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:49.322319 20802 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:49.322522 20537 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:49.322614 20406 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:49.324939 20142 consensus_queue.cc:237] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:49.326741 20935 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:49.327113 23593 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.327572 20802 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:49.327776 20272 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:49.327934 23593 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.328073 20539 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:49.328114 23594 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.328776 20406 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:49.329133 23602 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.329382 23593 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.329793 20668 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:49.330840 23593 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 07:59:49.337955 20870 consensus_peers.cc:485] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 15bb9480e88b41219cc91168cc438c70. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:49.338037 23594 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:49.339102 20406 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:49.339200 20275 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:49.340773 20935 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:49.340929 20802 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:49.341349 20001 catalog_manager.cc:5484] T 880876d792554606a7ccf21f9ec08e0f P 35642aa2099c40e59bc2092604078e07 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "29a7180fe4574c09a8dce7c27500aab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:49.341498 20668 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:49.346956 20142 consensus_queue.cc:237] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } }
I20240502 07:59:49.348816 20406 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:49.349412 20935 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:49.349442 20802 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:49.349474 20275 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:49.349762 20537 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:49.349807 23593 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:49.349900 23602 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:49.350008 23075 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:49.350119 20668 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:49.350423 23593 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:49.350664 23593 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.351356 23384 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:49.351315 20537 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:49.355479 23593 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:49.358366 20406 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:49.358496 20935 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:49.358803 19986 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 880876d792554606a7ccf21f9ec08e0f with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240502 07:59:49.359445 19993 catalog_manager.cc:5484] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 reported cstate change: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New cstate: current_term: 1 leader_uuid: "29a7180fe4574c09a8dce7c27500aab2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20240502 07:59:49.361014 20078 consensus_peers.cc:485] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 880876d792554606a7ccf21f9ec08e0f. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:49.360899 20275 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:49.361217 20537 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:49.367504 20668 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:49.371227 20802 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:49.484179 20406 consensus_queue.cc:237] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } }
I20240502 07:59:49.485900 20537 raft_consensus.cc:1261] T 04659b5e3504411987dafc49a5fa5ea7 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:49.485906 20668 raft_consensus.cc:1261] T 04659b5e3504411987dafc49a5fa5ea7 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:49.485906 22741 raft_consensus.cc:1261] T 04659b5e3504411987dafc49a5fa5ea7 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:49.485900 20802 raft_consensus.cc:1261] T 04659b5e3504411987dafc49a5fa5ea7 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:49.486361 23505 consensus_queue.cc:1035] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:49.486416 20142 raft_consensus.cc:1261] T 04659b5e3504411987dafc49a5fa5ea7 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:49.486451 20275 raft_consensus.cc:1261] T 04659b5e3504411987dafc49a5fa5ea7 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:49.486670 23505 consensus_queue.cc:1035] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:49.487074 23505 consensus_queue.cc:1035] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:49.487295 23728 consensus_queue.cc:1035] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:49.488008 23505 consensus_queue.cc:1035] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:49.488404 23727 consensus_queue.cc:1035] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:49.489219 23728 raft_consensus.cc:2932] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:49.489607 22741 raft_consensus.cc:2932] T 04659b5e3504411987dafc49a5fa5ea7 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:49.489791 20142 raft_consensus.cc:2932] T 04659b5e3504411987dafc49a5fa5ea7 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:49.490440 20275 raft_consensus.cc:2932] T 04659b5e3504411987dafc49a5fa5ea7 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:49.491356 20802 raft_consensus.cc:2932] T 04659b5e3504411987dafc49a5fa5ea7 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:49.491986 19988 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 04659b5e3504411987dafc49a5fa5ea7 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:49.492702 20000 catalog_manager.cc:5484] T 04659b5e3504411987dafc49a5fa5ea7 P 29a7180fe4574c09a8dce7c27500aab2 reported cstate change: config changed from index 6 to 7, VOTER b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71) evicted. New cstate: current_term: 1 leader_uuid: "35642aa2099c40e59bc2092604078e07" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:49.493731 20537 raft_consensus.cc:2932] T 04659b5e3504411987dafc49a5fa5ea7 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:49.495608 20668 raft_consensus.cc:2932] T 04659b5e3504411987dafc49a5fa5ea7 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:49.496420 20915 tablet_service.cc:1514] Processing DeleteTablet for tablet 04659b5e3504411987dafc49a5fa5ea7 with delete_type TABLET_DATA_TOMBSTONED (TS b5f471f8b5944986a7d878cf6ac550b8 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:41530
I20240502 07:59:49.497581 23730 tablet_replica.cc:330] T 04659b5e3504411987dafc49a5fa5ea7 P b5f471f8b5944986a7d878cf6ac550b8: stopping tablet replica
I20240502 07:59:49.497936 23730 raft_consensus.cc:2229] T 04659b5e3504411987dafc49a5fa5ea7 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:49.498250 23730 raft_consensus.cc:2258] T 04659b5e3504411987dafc49a5fa5ea7 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:49.499723 23730 ts_tablet_manager.cc:1875] T 04659b5e3504411987dafc49a5fa5ea7 P b5f471f8b5944986a7d878cf6ac550b8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:49.501582 23730 ts_tablet_manager.cc:1888] T 04659b5e3504411987dafc49a5fa5ea7 P b5f471f8b5944986a7d878cf6ac550b8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 07:59:49.501698 23730 log.cc:1196] T 04659b5e3504411987dafc49a5fa5ea7 P b5f471f8b5944986a7d878cf6ac550b8: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/wal/wals/04659b5e3504411987dafc49a5fa5ea7
I20240502 07:59:49.502245 19988 catalog_manager.cc:4830] TS b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577): tablet 04659b5e3504411987dafc49a5fa5ea7 (table rebalance_test_table_0 [id=89c0cb0099e94f1aa93c3eb299e446f0]) successfully deleted
I20240502 07:59:49.543778 20935 consensus_queue.cc:237] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } }
I20240502 07:59:49.545420 22475 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:49.545480 20142 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:49.545786 20537 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:49.545804 23560 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:49.545997 23383 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:49.546300 23383 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:49.547006 20802 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:49.547644 23382 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:49.548408 20275 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:49.548408 23559 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:49.548926 20537 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:49.549109 23561 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:49.549356 20406 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:49.549937 23557 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:49.550102 20275 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:49.550922 20406 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:49.551522 19988 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet f588a3ec49a24b1a9a9e19cdf8c30ca5 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:49.551533 22475 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:49.552109 19994 catalog_manager.cc:5484] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 reported cstate change: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New cstate: current_term: 1 leader_uuid: "b5f471f8b5944986a7d878cf6ac550b8" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:49.554330 20802 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:49.554872 20142 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:49.557508 20649 tablet_service.cc:1514] Processing DeleteTablet for tablet f588a3ec49a24b1a9a9e19cdf8c30ca5 with delete_type TABLET_DATA_TOMBSTONED (TS b443e68b6212450286869124684bdfff not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:53090
I20240502 07:59:49.557827 23731 tablet_replica.cc:330] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b443e68b6212450286869124684bdfff: stopping tablet replica
I20240502 07:59:49.557955 23731 raft_consensus.cc:2229] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:49.558151 23731 raft_consensus.cc:2258] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:49.558908 23731 ts_tablet_manager.cc:1875] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b443e68b6212450286869124684bdfff: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:49.560649 23731 ts_tablet_manager.cc:1888] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b443e68b6212450286869124684bdfff: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 07:59:49.560709 23731 log.cc:1196] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b443e68b6212450286869124684bdfff: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/wal/wals/f588a3ec49a24b1a9a9e19cdf8c30ca5
I20240502 07:59:49.561100 19987 catalog_manager.cc:4830] TS b443e68b6212450286869124684bdfff (127.3.139.69:37923): tablet f588a3ec49a24b1a9a9e19cdf8c30ca5 (table rebalance_test_table_2 [id=30a30b02a2404507a95fc3b5bb77bdd1]) successfully deleted
I20240502 07:59:49.630945 20668 consensus_queue.cc:237] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:49.632577 20935 raft_consensus.cc:1261] T c3ca6f24c92d4273ac6c9d83161813d7 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:49.632541 20406 raft_consensus.cc:1261] T c3ca6f24c92d4273ac6c9d83161813d7 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:49.632647 20537 raft_consensus.cc:1261] T c3ca6f24c92d4273ac6c9d83161813d7 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:49.632645 20142 raft_consensus.cc:1261] T c3ca6f24c92d4273ac6c9d83161813d7 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:49.632879 20802 raft_consensus.cc:1261] T c3ca6f24c92d4273ac6c9d83161813d7 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:49.633242 23573 consensus_queue.cc:1035] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.633390 23574 consensus_queue.cc:1035] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.633548 23099 consensus_queue.cc:1035] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.633646 20275 raft_consensus.cc:1261] T c3ca6f24c92d4273ac6c9d83161813d7 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:49.633677 23274 consensus_queue.cc:1035] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.633788 23571 consensus_queue.cc:1035] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.634297 23573 consensus_queue.cc:1035] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.636186 23099 raft_consensus.cc:2932] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:49.637457 20142 raft_consensus.cc:2932] T c3ca6f24c92d4273ac6c9d83161813d7 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:49.638299 20933 raft_consensus.cc:2932] T c3ca6f24c92d4273ac6c9d83161813d7 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:49.639093 20537 raft_consensus.cc:2932] T c3ca6f24c92d4273ac6c9d83161813d7 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:49.640434 20802 raft_consensus.cc:2932] T c3ca6f24c92d4273ac6c9d83161813d7 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:49.641170 20275 raft_consensus.cc:2932] T c3ca6f24c92d4273ac6c9d83161813d7 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:49.641912 19997 catalog_manager.cc:5484] T c3ca6f24c92d4273ac6c9d83161813d7 P 29a7180fe4574c09a8dce7c27500aab2 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "b443e68b6212450286869124684bdfff" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:49.645494 23748 ts_tablet_manager.cc:902] T a973de06330c487ba54a8a36b1e31b54 P f014c4ad4c1342219a1004181ac33ed8: Initiating tablet copy from peer e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315)
I20240502 07:59:49.646143 23748 tablet_copy_client.cc:308] T a973de06330c487ba54a8a36b1e31b54 P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.68:40315
I20240502 07:59:49.647446 20668 consensus_queue.cc:237] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } }
I20240502 07:59:49.647368 20406 raft_consensus.cc:2932] T c3ca6f24c92d4273ac6c9d83161813d7 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:49.649142 20275 raft_consensus.cc:1261] T c3ca6f24c92d4273ac6c9d83161813d7 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20240502 07:59:49.649498 20606 consensus_peers.cc:485] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c3ca6f24c92d4273ac6c9d83161813d7. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:49.649731 20406 raft_consensus.cc:1261] T c3ca6f24c92d4273ac6c9d83161813d7 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:49.649778 23572 consensus_queue.cc:1035] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:49.650290 23572 consensus_queue.cc:1035] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:49.650852 20933 raft_consensus.cc:1261] T c3ca6f24c92d4273ac6c9d83161813d7 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:49.651631 20142 raft_consensus.cc:1261] T c3ca6f24c92d4273ac6c9d83161813d7 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:49.651906 23571 consensus_queue.cc:1035] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:49.652452 20537 raft_consensus.cc:1261] T c3ca6f24c92d4273ac6c9d83161813d7 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:49.652837 20802 raft_consensus.cc:1261] T c3ca6f24c92d4273ac6c9d83161813d7 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:49.653173 23571 consensus_queue.cc:1035] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:49.653348 23572 consensus_queue.cc:1035] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:49.653730 23574 consensus_queue.cc:1035] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:49.655505 20668 consensus_queue.cc:237] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:49.656227 23574 raft_consensus.cc:2932] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:49.656669 20933 raft_consensus.cc:2932] T c3ca6f24c92d4273ac6c9d83161813d7 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:49.657963 20142 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:49.658046 20935 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:49.658481 23573 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.658444 19997 catalog_manager.cc:5484] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff reported cstate change: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New cstate: current_term: 1 leader_uuid: "b443e68b6212450286869124684bdfff" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:49.658689 20142 raft_consensus.cc:2932] T c3ca6f24c92d4273ac6c9d83161813d7 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:49.659097 20802 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:49.659300 20272 raft_consensus.cc:2932] T c3ca6f24c92d4273ac6c9d83161813d7 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:49.659490 20406 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:49.660588 23571 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.661834 20802 raft_consensus.cc:2932] T c3ca6f24c92d4273ac6c9d83161813d7 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:49.662225 20539 raft_consensus.cc:2932] T c3ca6f24c92d4273ac6c9d83161813d7 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:49.664294 20537 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:49.664788 20275 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:49.664934 23099 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.665302 19987 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet c3ca6f24c92d4273ac6c9d83161813d7 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240502 07:59:49.665354 23099 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.665668 20406 raft_consensus.cc:2932] T c3ca6f24c92d4273ac6c9d83161813d7 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:49.665951 23574 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.666239 23574 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.667043 20559 tablet_copy_service.cc:140] P e951f009258e473cb57cf8773c7975d4: Received BeginTabletCopySession request for tablet a973de06330c487ba54a8a36b1e31b54 from peer f014c4ad4c1342219a1004181ac33ed8 ({username='slave'} at 127.3.139.72:45019)
I20240502 07:59:49.667181 20559 tablet_copy_service.cc:161] P e951f009258e473cb57cf8773c7975d4: Beginning new tablet copy session on tablet a973de06330c487ba54a8a36b1e31b54 from peer f014c4ad4c1342219a1004181ac33ed8 at {username='slave'} at 127.3.139.72:45019: session id = f014c4ad4c1342219a1004181ac33ed8-a973de06330c487ba54a8a36b1e31b54
I20240502 07:59:49.668408 20559 tablet_copy_source_session.cc:205] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:49.669508 23748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a973de06330c487ba54a8a36b1e31b54. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:49.671563 23748 tablet_copy_client.cc:783] T a973de06330c487ba54a8a36b1e31b54 P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:49.671697 23748 tablet_copy_client.cc:647] T a973de06330c487ba54a8a36b1e31b54 P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:49.672160 23099 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:49.674172 20933 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:49.674412 19994 catalog_manager.cc:5484] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "b443e68b6212450286869124684bdfff" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: HEALTHY } } }
I20240502 07:59:49.675513 20405 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:49.677476 20800 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:49.678802 20142 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:49.680292 20668 consensus_queue.cc:237] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } }
I20240502 07:59:49.682503 20275 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:49.682880 23573 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 07:59:49.683349 20606 consensus_peers.cc:485] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1bc5937510214c3da7731d3ad2a67842. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:49.683756 20933 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:49.684119 20405 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:49.684365 20800 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:49.684628 20537 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:49.685545 23755 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:49.685812 23755 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:49.685809 20275 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:49.686959 23748 tablet_copy_client.cc:523] T a973de06330c487ba54a8a36b1e31b54 P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:49.688081 23755 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.688812 20537 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:49.689970 23748 tablet_bootstrap.cc:492] T a973de06330c487ba54a8a36b1e31b54 P f014c4ad4c1342219a1004181ac33ed8: Bootstrap starting.
I20240502 07:59:49.690286 23099 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:49.690699 20142 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:49.692221 23573 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:49.692497 23573 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:49.694988 20405 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:49.694985 20933 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:49.695154 19997 catalog_manager.cc:5484] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff reported cstate change: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New cstate: current_term: 1 leader_uuid: "b443e68b6212450286869124684bdfff" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:49.696887 19987 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 1bc5937510214c3da7731d3ad2a67842 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240502 07:59:49.698407 20537 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:49.699206 23768 ts_tablet_manager.cc:902] T b73e0ce533014420af513429620af68c P ff9f2ccccdfa48a1afbc0e371f41c662: Initiating tablet copy from peer c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085)
I20240502 07:59:49.699764 23768 tablet_copy_client.cc:308] T b73e0ce533014420af513429620af68c P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.70:36085
I20240502 07:59:49.700351 23771 ts_tablet_manager.cc:902] T bbae35bae8c047a6982e128ffb77ca86 P ff9f2ccccdfa48a1afbc0e371f41c662: Initiating tablet copy from peer b443e68b6212450286869124684bdfff (127.3.139.69:37923)
I20240502 07:59:49.700271 20800 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:49.700925 23771 tablet_copy_client.cc:308] T bbae35bae8c047a6982e128ffb77ca86 P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.69:37923
I20240502 07:59:49.701436 20823 tablet_copy_service.cc:140] P c15e23a34059456788f9775031e27ec0: Received BeginTabletCopySession request for tablet b73e0ce533014420af513429620af68c from peer ff9f2ccccdfa48a1afbc0e371f41c662 ({username='slave'} at 127.3.139.74:42219)
I20240502 07:59:49.701507 20823 tablet_copy_service.cc:161] P c15e23a34059456788f9775031e27ec0: Beginning new tablet copy session on tablet b73e0ce533014420af513429620af68c from peer ff9f2ccccdfa48a1afbc0e371f41c662 at {username='slave'} at 127.3.139.74:42219: session id = ff9f2ccccdfa48a1afbc0e371f41c662-b73e0ce533014420af513429620af68c
I20240502 07:59:49.702297 20823 tablet_copy_source_session.cc:205] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:49.702302 20142 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:49.703109 23768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b73e0ce533014420af513429620af68c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:49.705566 20275 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:49.705857 23768 tablet_copy_client.cc:783] T b73e0ce533014420af513429620af68c P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:49.706931 23768 tablet_copy_client.cc:647] T b73e0ce533014420af513429620af68c P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:49.708092 23768 tablet_copy_client.cc:523] T b73e0ce533014420af513429620af68c P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:49.709477 23768 tablet_bootstrap.cc:492] T b73e0ce533014420af513429620af68c P ff9f2ccccdfa48a1afbc0e371f41c662: Bootstrap starting.
I20240502 07:59:49.711177 20691 tablet_copy_service.cc:140] P b443e68b6212450286869124684bdfff: Received BeginTabletCopySession request for tablet bbae35bae8c047a6982e128ffb77ca86 from peer ff9f2ccccdfa48a1afbc0e371f41c662 ({username='slave'} at 127.3.139.74:38165)
I20240502 07:59:49.711266 20691 tablet_copy_service.cc:161] P b443e68b6212450286869124684bdfff: Beginning new tablet copy session on tablet bbae35bae8c047a6982e128ffb77ca86 from peer ff9f2ccccdfa48a1afbc0e371f41c662 at {username='slave'} at 127.3.139.74:38165: session id = ff9f2ccccdfa48a1afbc0e371f41c662-bbae35bae8c047a6982e128ffb77ca86
I20240502 07:59:49.711998 20691 tablet_copy_source_session.cc:205] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:49.712929 23771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbae35bae8c047a6982e128ffb77ca86. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:49.714723 23771 tablet_copy_client.cc:783] T bbae35bae8c047a6982e128ffb77ca86 P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:49.714865 23771 tablet_copy_client.cc:647] T bbae35bae8c047a6982e128ffb77ca86 P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:49.715957 23771 tablet_copy_client.cc:523] T bbae35bae8c047a6982e128ffb77ca86 P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:49.716859 23771 tablet_bootstrap.cc:492] T bbae35bae8c047a6982e128ffb77ca86 P ff9f2ccccdfa48a1afbc0e371f41c662: Bootstrap starting.
I20240502 07:59:49.739398 23768 tablet_bootstrap.cc:492] T b73e0ce533014420af513429620af68c P ff9f2ccccdfa48a1afbc0e371f41c662: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:49.739737 23768 tablet_bootstrap.cc:492] T b73e0ce533014420af513429620af68c P ff9f2ccccdfa48a1afbc0e371f41c662: Bootstrap complete.
I20240502 07:59:49.739917 23768 ts_tablet_manager.cc:1367] T b73e0ce533014420af513429620af68c P ff9f2ccccdfa48a1afbc0e371f41c662: Time spent bootstrapping tablet: real 0.030s user 0.023s sys 0.003s
I20240502 07:59:49.740471 23768 raft_consensus.cc:348] T b73e0ce533014420af513429620af68c P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } }
I20240502 07:59:49.740605 23768 raft_consensus.cc:729] T b73e0ce533014420af513429620af68c P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ff9f2ccccdfa48a1afbc0e371f41c662, State: Initialized, Role: LEARNER
I20240502 07:59:49.740761 23768 consensus_queue.cc:260] T b73e0ce533014420af513429620af68c P ff9f2ccccdfa48a1afbc0e371f41c662 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } }
I20240502 07:59:49.741134 23768 ts_tablet_manager.cc:1398] T b73e0ce533014420af513429620af68c P ff9f2ccccdfa48a1afbc0e371f41c662: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:49.741583 20823 tablet_copy_service.cc:342] P c15e23a34059456788f9775031e27ec0: Request end of tablet copy session ff9f2ccccdfa48a1afbc0e371f41c662-b73e0ce533014420af513429620af68c received from {username='slave'} at 127.3.139.74:42219
I20240502 07:59:49.741657 20823 tablet_copy_service.cc:433] P c15e23a34059456788f9775031e27ec0: ending tablet copy session ff9f2ccccdfa48a1afbc0e371f41c662-b73e0ce533014420af513429620af68c on tablet b73e0ce533014420af513429620af68c with peer ff9f2ccccdfa48a1afbc0e371f41c662
I20240502 07:59:49.748874 23748 tablet_bootstrap.cc:492] T a973de06330c487ba54a8a36b1e31b54 P f014c4ad4c1342219a1004181ac33ed8: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:49.749106 23748 tablet_bootstrap.cc:492] T a973de06330c487ba54a8a36b1e31b54 P f014c4ad4c1342219a1004181ac33ed8: Bootstrap complete.
I20240502 07:59:49.749259 23748 ts_tablet_manager.cc:1367] T a973de06330c487ba54a8a36b1e31b54 P f014c4ad4c1342219a1004181ac33ed8: Time spent bootstrapping tablet: real 0.059s user 0.020s sys 0.002s
I20240502 07:59:49.749590 23748 raft_consensus.cc:348] T a973de06330c487ba54a8a36b1e31b54 P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } }
I20240502 07:59:49.749691 23748 raft_consensus.cc:729] T a973de06330c487ba54a8a36b1e31b54 P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f014c4ad4c1342219a1004181ac33ed8, State: Initialized, Role: LEARNER
I20240502 07:59:49.749776 23748 consensus_queue.cc:260] T a973de06330c487ba54a8a36b1e31b54 P f014c4ad4c1342219a1004181ac33ed8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } }
I20240502 07:59:49.750007 23748 ts_tablet_manager.cc:1398] T a973de06330c487ba54a8a36b1e31b54 P f014c4ad4c1342219a1004181ac33ed8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:49.750723 20559 tablet_copy_service.cc:342] P e951f009258e473cb57cf8773c7975d4: Request end of tablet copy session f014c4ad4c1342219a1004181ac33ed8-a973de06330c487ba54a8a36b1e31b54 received from {username='slave'} at 127.3.139.72:45019
I20240502 07:59:49.750816 20559 tablet_copy_service.cc:433] P e951f009258e473cb57cf8773c7975d4: ending tablet copy session f014c4ad4c1342219a1004181ac33ed8-a973de06330c487ba54a8a36b1e31b54 on tablet a973de06330c487ba54a8a36b1e31b54 with peer f014c4ad4c1342219a1004181ac33ed8
I20240502 07:59:49.753309 23771 tablet_bootstrap.cc:492] T bbae35bae8c047a6982e128ffb77ca86 P ff9f2ccccdfa48a1afbc0e371f41c662: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:49.753504 23771 tablet_bootstrap.cc:492] T bbae35bae8c047a6982e128ffb77ca86 P ff9f2ccccdfa48a1afbc0e371f41c662: Bootstrap complete.
I20240502 07:59:49.753641 23771 ts_tablet_manager.cc:1367] T bbae35bae8c047a6982e128ffb77ca86 P ff9f2ccccdfa48a1afbc0e371f41c662: Time spent bootstrapping tablet: real 0.037s user 0.019s sys 0.000s
I20240502 07:59:49.753940 23771 raft_consensus.cc:348] T bbae35bae8c047a6982e128ffb77ca86 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } }
I20240502 07:59:49.754046 23771 raft_consensus.cc:729] T bbae35bae8c047a6982e128ffb77ca86 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ff9f2ccccdfa48a1afbc0e371f41c662, State: Initialized, Role: LEARNER
I20240502 07:59:49.754155 23771 consensus_queue.cc:260] T bbae35bae8c047a6982e128ffb77ca86 P ff9f2ccccdfa48a1afbc0e371f41c662 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } }
I20240502 07:59:49.754395 23771 ts_tablet_manager.cc:1398] T bbae35bae8c047a6982e128ffb77ca86 P ff9f2ccccdfa48a1afbc0e371f41c662: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:49.754900 20691 tablet_copy_service.cc:342] P b443e68b6212450286869124684bdfff: Request end of tablet copy session ff9f2ccccdfa48a1afbc0e371f41c662-bbae35bae8c047a6982e128ffb77ca86 received from {username='slave'} at 127.3.139.74:38165
I20240502 07:59:49.754967 20691 tablet_copy_service.cc:433] P b443e68b6212450286869124684bdfff: ending tablet copy session ff9f2ccccdfa48a1afbc0e371f41c662-bbae35bae8c047a6982e128ffb77ca86 on tablet bbae35bae8c047a6982e128ffb77ca86 with peer ff9f2ccccdfa48a1afbc0e371f41c662
I20240502 07:59:49.804687 23771 ts_tablet_manager.cc:902] T 880876d792554606a7ccf21f9ec08e0f P ff9f2ccccdfa48a1afbc0e371f41c662: Initiating tablet copy from peer 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703)
I20240502 07:59:49.805004 23771 tablet_copy_client.cc:308] T 880876d792554606a7ccf21f9ec08e0f P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.65:44703
I20240502 07:59:49.808193 20163 tablet_copy_service.cc:140] P 29a7180fe4574c09a8dce7c27500aab2: Received BeginTabletCopySession request for tablet 880876d792554606a7ccf21f9ec08e0f from peer ff9f2ccccdfa48a1afbc0e371f41c662 ({username='slave'} at 127.3.139.74:47167)
I20240502 07:59:49.808283 20163 tablet_copy_service.cc:161] P 29a7180fe4574c09a8dce7c27500aab2: Beginning new tablet copy session on tablet 880876d792554606a7ccf21f9ec08e0f from peer ff9f2ccccdfa48a1afbc0e371f41c662 at {username='slave'} at 127.3.139.74:47167: session id = ff9f2ccccdfa48a1afbc0e371f41c662-880876d792554606a7ccf21f9ec08e0f
I20240502 07:59:49.808974 20163 tablet_copy_source_session.cc:205] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:49.809684 23771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 880876d792554606a7ccf21f9ec08e0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:49.811213 23771 tablet_copy_client.cc:783] T 880876d792554606a7ccf21f9ec08e0f P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:49.811342 23771 tablet_copy_client.cc:647] T 880876d792554606a7ccf21f9ec08e0f P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:49.812345 23771 tablet_copy_client.cc:523] T 880876d792554606a7ccf21f9ec08e0f P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:49.813206 23771 tablet_bootstrap.cc:492] T 880876d792554606a7ccf21f9ec08e0f P ff9f2ccccdfa48a1afbc0e371f41c662: Bootstrap starting.
I20240502 07:59:49.817559 23768 ts_tablet_manager.cc:902] T 15bb9480e88b41219cc91168cc438c70 P ff9f2ccccdfa48a1afbc0e371f41c662: Initiating tablet copy from peer b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577)
I20240502 07:59:49.817942 23768 tablet_copy_client.cc:308] T 15bb9480e88b41219cc91168cc438c70 P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.71:39577
I20240502 07:59:49.821278 20955 tablet_copy_service.cc:140] P b5f471f8b5944986a7d878cf6ac550b8: Received BeginTabletCopySession request for tablet 15bb9480e88b41219cc91168cc438c70 from peer ff9f2ccccdfa48a1afbc0e371f41c662 ({username='slave'} at 127.3.139.74:54441)
I20240502 07:59:49.821372 20955 tablet_copy_service.cc:161] P b5f471f8b5944986a7d878cf6ac550b8: Beginning new tablet copy session on tablet 15bb9480e88b41219cc91168cc438c70 from peer ff9f2ccccdfa48a1afbc0e371f41c662 at {username='slave'} at 127.3.139.74:54441: session id = ff9f2ccccdfa48a1afbc0e371f41c662-15bb9480e88b41219cc91168cc438c70
I20240502 07:59:49.822173 20955 tablet_copy_source_session.cc:205] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:49.822863 23768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15bb9480e88b41219cc91168cc438c70. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:49.824303 23768 tablet_copy_client.cc:783] T 15bb9480e88b41219cc91168cc438c70 P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:49.824433 23768 tablet_copy_client.cc:647] T 15bb9480e88b41219cc91168cc438c70 P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:49.825512 23768 tablet_copy_client.cc:523] T 15bb9480e88b41219cc91168cc438c70 P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:49.826341 23768 tablet_bootstrap.cc:492] T 15bb9480e88b41219cc91168cc438c70 P ff9f2ccccdfa48a1afbc0e371f41c662: Bootstrap starting.
I20240502 07:59:49.832645 23771 tablet_bootstrap.cc:492] T 880876d792554606a7ccf21f9ec08e0f P ff9f2ccccdfa48a1afbc0e371f41c662: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:49.832876 23771 tablet_bootstrap.cc:492] T 880876d792554606a7ccf21f9ec08e0f P ff9f2ccccdfa48a1afbc0e371f41c662: Bootstrap complete.
I20240502 07:59:49.833048 23771 ts_tablet_manager.cc:1367] T 880876d792554606a7ccf21f9ec08e0f P ff9f2ccccdfa48a1afbc0e371f41c662: Time spent bootstrapping tablet: real 0.020s user 0.020s sys 0.000s
I20240502 07:59:49.833410 23771 raft_consensus.cc:348] T 880876d792554606a7ccf21f9ec08e0f P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } }
I20240502 07:59:49.833534 23771 raft_consensus.cc:729] T 880876d792554606a7ccf21f9ec08e0f P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ff9f2ccccdfa48a1afbc0e371f41c662, State: Initialized, Role: LEARNER
I20240502 07:59:49.833670 23771 consensus_queue.cc:260] T 880876d792554606a7ccf21f9ec08e0f P ff9f2ccccdfa48a1afbc0e371f41c662 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } }
I20240502 07:59:49.833907 23771 ts_tablet_manager.cc:1398] T 880876d792554606a7ccf21f9ec08e0f P ff9f2ccccdfa48a1afbc0e371f41c662: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:49.834339 20163 tablet_copy_service.cc:342] P 29a7180fe4574c09a8dce7c27500aab2: Request end of tablet copy session ff9f2ccccdfa48a1afbc0e371f41c662-880876d792554606a7ccf21f9ec08e0f received from {username='slave'} at 127.3.139.74:47167
I20240502 07:59:49.834409 20163 tablet_copy_service.cc:433] P 29a7180fe4574c09a8dce7c27500aab2: ending tablet copy session ff9f2ccccdfa48a1afbc0e371f41c662-880876d792554606a7ccf21f9ec08e0f on tablet 880876d792554606a7ccf21f9ec08e0f with peer ff9f2ccccdfa48a1afbc0e371f41c662
I20240502 07:59:49.844270 23768 tablet_bootstrap.cc:492] T 15bb9480e88b41219cc91168cc438c70 P ff9f2ccccdfa48a1afbc0e371f41c662: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:49.844494 23768 tablet_bootstrap.cc:492] T 15bb9480e88b41219cc91168cc438c70 P ff9f2ccccdfa48a1afbc0e371f41c662: Bootstrap complete.
I20240502 07:59:49.844638 23768 ts_tablet_manager.cc:1367] T 15bb9480e88b41219cc91168cc438c70 P ff9f2ccccdfa48a1afbc0e371f41c662: Time spent bootstrapping tablet: real 0.018s user 0.014s sys 0.003s
I20240502 07:59:49.844961 23768 raft_consensus.cc:348] T 15bb9480e88b41219cc91168cc438c70 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } }
I20240502 07:59:49.845067 23768 raft_consensus.cc:729] T 15bb9480e88b41219cc91168cc438c70 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ff9f2ccccdfa48a1afbc0e371f41c662, State: Initialized, Role: LEARNER
I20240502 07:59:49.845146 23768 consensus_queue.cc:260] T 15bb9480e88b41219cc91168cc438c70 P ff9f2ccccdfa48a1afbc0e371f41c662 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } }
I20240502 07:59:49.845400 23768 ts_tablet_manager.cc:1398] T 15bb9480e88b41219cc91168cc438c70 P ff9f2ccccdfa48a1afbc0e371f41c662: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:49.845786 20955 tablet_copy_service.cc:342] P b5f471f8b5944986a7d878cf6ac550b8: Request end of tablet copy session ff9f2ccccdfa48a1afbc0e371f41c662-15bb9480e88b41219cc91168cc438c70 received from {username='slave'} at 127.3.139.74:54441
I20240502 07:59:49.845870 20955 tablet_copy_service.cc:433] P b5f471f8b5944986a7d878cf6ac550b8: ending tablet copy session ff9f2ccccdfa48a1afbc0e371f41c662-15bb9480e88b41219cc91168cc438c70 on tablet 15bb9480e88b41219cc91168cc438c70 with peer ff9f2ccccdfa48a1afbc0e371f41c662
I20240502 07:59:49.985359 20275 consensus_queue.cc:237] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:49.987007 20668 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:49.987298 20800 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:49.987443 20537 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:49.987581 20406 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:49.987746 20142 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:49.987841 23844 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.988050 20933 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:49.988057 23797 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.988327 23844 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.988607 23844 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.988756 22993 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.989028 23844 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:49.992753 22993 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:49.993228 20537 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:49.993592 20142 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:49.994918 20933 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:49.995016 20406 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:49.995798 19994 catalog_manager.cc:5484] T ae6d597fbc9242539edee8685a015d99 P e951f009258e473cb57cf8773c7975d4 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:49.997893 20800 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:49.999389 20668 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:50.001559 20275 consensus_queue.cc:237] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } }
I20240502 07:59:50.003376 20800 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:50.003438 20142 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:50.003751 20537 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:50.003731 23844 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:50.004026 23797 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:50.004384 20933 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:50.004384 20406 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:50.004773 23797 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:50.004947 20668 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:50.005086 23843 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:50.005311 23842 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:50.006523 23844 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:50.006973 23844 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:50.007360 20406 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:50.008971 19987 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet ae6d597fbc9242539edee8685a015d99 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240502 07:59:50.009020 20933 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:50.009801 20800 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:50.009881 20001 catalog_manager.cc:5484] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 reported cstate change: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New cstate: current_term: 1 leader_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:50.010274 20537 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
W20240502 07:59:50.013453 20210 consensus_peers.cc:485] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ae6d597fbc9242539edee8685a015d99. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:50.013847 20142 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:50.014125 20668 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:50.129555 22741 raft_consensus.cc:1203] T a973de06330c487ba54a8a36b1e31b54 P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240502 07:59:50.182480 22609 raft_consensus.cc:1203] T 880876d792554606a7ccf21f9ec08e0f P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240502 07:59:50.184118 23768 ts_tablet_manager.cc:902] T 1bc5937510214c3da7731d3ad2a67842 P ff9f2ccccdfa48a1afbc0e371f41c662: Initiating tablet copy from peer b443e68b6212450286869124684bdfff (127.3.139.69:37923)
I20240502 07:59:50.184540 23768 tablet_copy_client.cc:308] T 1bc5937510214c3da7731d3ad2a67842 P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.69:37923
I20240502 07:59:50.184937 20691 tablet_copy_service.cc:140] P b443e68b6212450286869124684bdfff: Received BeginTabletCopySession request for tablet 1bc5937510214c3da7731d3ad2a67842 from peer ff9f2ccccdfa48a1afbc0e371f41c662 ({username='slave'} at 127.3.139.74:38165)
I20240502 07:59:50.185036 20691 tablet_copy_service.cc:161] P b443e68b6212450286869124684bdfff: Beginning new tablet copy session on tablet 1bc5937510214c3da7731d3ad2a67842 from peer ff9f2ccccdfa48a1afbc0e371f41c662 at {username='slave'} at 127.3.139.74:38165: session id = ff9f2ccccdfa48a1afbc0e371f41c662-1bc5937510214c3da7731d3ad2a67842
I20240502 07:59:50.185664 20691 tablet_copy_source_session.cc:205] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:50.186451 23768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bc5937510214c3da7731d3ad2a67842. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:50.188107 23768 tablet_copy_client.cc:783] T 1bc5937510214c3da7731d3ad2a67842 P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:50.188248 23768 tablet_copy_client.cc:647] T 1bc5937510214c3da7731d3ad2a67842 P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:50.189184 23768 tablet_copy_client.cc:523] T 1bc5937510214c3da7731d3ad2a67842 P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:50.190053 23768 tablet_bootstrap.cc:492] T 1bc5937510214c3da7731d3ad2a67842 P ff9f2ccccdfa48a1afbc0e371f41c662: Bootstrap starting.
I20240502 07:59:50.213683 23768 tablet_bootstrap.cc:492] T 1bc5937510214c3da7731d3ad2a67842 P ff9f2ccccdfa48a1afbc0e371f41c662: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:50.213932 23768 tablet_bootstrap.cc:492] T 1bc5937510214c3da7731d3ad2a67842 P ff9f2ccccdfa48a1afbc0e371f41c662: Bootstrap complete.
I20240502 07:59:50.214134 23768 ts_tablet_manager.cc:1367] T 1bc5937510214c3da7731d3ad2a67842 P ff9f2ccccdfa48a1afbc0e371f41c662: Time spent bootstrapping tablet: real 0.024s user 0.024s sys 0.000s
I20240502 07:59:50.214541 23768 raft_consensus.cc:348] T 1bc5937510214c3da7731d3ad2a67842 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } }
I20240502 07:59:50.214682 23768 raft_consensus.cc:729] T 1bc5937510214c3da7731d3ad2a67842 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ff9f2ccccdfa48a1afbc0e371f41c662, State: Initialized, Role: LEARNER
I20240502 07:59:50.214771 23768 consensus_queue.cc:260] T 1bc5937510214c3da7731d3ad2a67842 P ff9f2ccccdfa48a1afbc0e371f41c662 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } }
I20240502 07:59:50.215533 23768 ts_tablet_manager.cc:1398] T 1bc5937510214c3da7731d3ad2a67842 P ff9f2ccccdfa48a1afbc0e371f41c662: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:50.216735 20691 tablet_copy_service.cc:342] P b443e68b6212450286869124684bdfff: Request end of tablet copy session ff9f2ccccdfa48a1afbc0e371f41c662-1bc5937510214c3da7731d3ad2a67842 received from {username='slave'} at 127.3.139.74:38165
I20240502 07:59:50.216820 20691 tablet_copy_service.cc:433] P b443e68b6212450286869124684bdfff: ending tablet copy session ff9f2ccccdfa48a1afbc0e371f41c662-1bc5937510214c3da7731d3ad2a67842 on tablet 1bc5937510214c3da7731d3ad2a67842 with peer ff9f2ccccdfa48a1afbc0e371f41c662
I20240502 07:59:50.228569 22609 raft_consensus.cc:1203] T 15bb9480e88b41219cc91168cc438c70 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240502 07:59:50.253664 23768 ts_tablet_manager.cc:902] T c3ca6f24c92d4273ac6c9d83161813d7 P ff9f2ccccdfa48a1afbc0e371f41c662: Initiating tablet copy from peer b443e68b6212450286869124684bdfff (127.3.139.69:37923)
I20240502 07:59:50.254102 23768 tablet_copy_client.cc:308] T c3ca6f24c92d4273ac6c9d83161813d7 P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.69:37923
I20240502 07:59:50.254523 20691 tablet_copy_service.cc:140] P b443e68b6212450286869124684bdfff: Received BeginTabletCopySession request for tablet c3ca6f24c92d4273ac6c9d83161813d7 from peer ff9f2ccccdfa48a1afbc0e371f41c662 ({username='slave'} at 127.3.139.74:38165)
I20240502 07:59:50.254634 20691 tablet_copy_service.cc:161] P b443e68b6212450286869124684bdfff: Beginning new tablet copy session on tablet c3ca6f24c92d4273ac6c9d83161813d7 from peer ff9f2ccccdfa48a1afbc0e371f41c662 at {username='slave'} at 127.3.139.74:38165: session id = ff9f2ccccdfa48a1afbc0e371f41c662-c3ca6f24c92d4273ac6c9d83161813d7
I20240502 07:59:50.255599 20691 tablet_copy_source_session.cc:205] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:50.256624 23768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3ca6f24c92d4273ac6c9d83161813d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:50.258709 23768 tablet_copy_client.cc:783] T c3ca6f24c92d4273ac6c9d83161813d7 P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:50.258852 23768 tablet_copy_client.cc:647] T c3ca6f24c92d4273ac6c9d83161813d7 P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:50.260386 23768 tablet_copy_client.cc:523] T c3ca6f24c92d4273ac6c9d83161813d7 P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:50.260835 22609 raft_consensus.cc:1203] T bbae35bae8c047a6982e128ffb77ca86 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240502 07:59:50.261487 23768 tablet_bootstrap.cc:492] T c3ca6f24c92d4273ac6c9d83161813d7 P ff9f2ccccdfa48a1afbc0e371f41c662: Bootstrap starting.
I20240502 07:59:50.280948 22609 raft_consensus.cc:1203] T b73e0ce533014420af513429620af68c P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240502 07:59:50.285459 23768 tablet_bootstrap.cc:492] T c3ca6f24c92d4273ac6c9d83161813d7 P ff9f2ccccdfa48a1afbc0e371f41c662: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:50.285671 23768 tablet_bootstrap.cc:492] T c3ca6f24c92d4273ac6c9d83161813d7 P ff9f2ccccdfa48a1afbc0e371f41c662: Bootstrap complete.
I20240502 07:59:50.285815 23768 ts_tablet_manager.cc:1367] T c3ca6f24c92d4273ac6c9d83161813d7 P ff9f2ccccdfa48a1afbc0e371f41c662: Time spent bootstrapping tablet: real 0.024s user 0.022s sys 0.000s
I20240502 07:59:50.286211 23768 raft_consensus.cc:348] T c3ca6f24c92d4273ac6c9d83161813d7 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } }
I20240502 07:59:50.286324 23768 raft_consensus.cc:729] T c3ca6f24c92d4273ac6c9d83161813d7 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ff9f2ccccdfa48a1afbc0e371f41c662, State: Initialized, Role: LEARNER
I20240502 07:59:50.286422 23768 consensus_queue.cc:260] T c3ca6f24c92d4273ac6c9d83161813d7 P ff9f2ccccdfa48a1afbc0e371f41c662 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } }
I20240502 07:59:50.286845 23768 ts_tablet_manager.cc:1398] T c3ca6f24c92d4273ac6c9d83161813d7 P ff9f2ccccdfa48a1afbc0e371f41c662: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:50.287218 20691 tablet_copy_service.cc:342] P b443e68b6212450286869124684bdfff: Request end of tablet copy session ff9f2ccccdfa48a1afbc0e371f41c662-c3ca6f24c92d4273ac6c9d83161813d7 received from {username='slave'} at 127.3.139.74:38165
I20240502 07:59:50.287295 20691 tablet_copy_service.cc:433] P b443e68b6212450286869124684bdfff: ending tablet copy session ff9f2ccccdfa48a1afbc0e371f41c662-c3ca6f24c92d4273ac6c9d83161813d7 on tablet c3ca6f24c92d4273ac6c9d83161813d7 with peer ff9f2ccccdfa48a1afbc0e371f41c662
I20240502 07:59:50.612530 23991 ts_tablet_manager.cc:902] T ae6d597fbc9242539edee8685a015d99 P f014c4ad4c1342219a1004181ac33ed8: Initiating tablet copy from peer b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331)
I20240502 07:59:50.612964 23991 tablet_copy_client.cc:308] T ae6d597fbc9242539edee8685a015d99 P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.66:45331
I20240502 07:59:50.616747 23625 raft_consensus.cc:1050] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4: attempting to promote NON_VOTER f014c4ad4c1342219a1004181ac33ed8 to VOTER
I20240502 07:59:50.617069 20295 tablet_copy_service.cc:140] P b7604c0eb3bf4a7cb260b564a94a1191: Received BeginTabletCopySession request for tablet ae6d597fbc9242539edee8685a015d99 from peer f014c4ad4c1342219a1004181ac33ed8 ({username='slave'} at 127.3.139.72:46325)
I20240502 07:59:50.617170 20295 tablet_copy_service.cc:161] P b7604c0eb3bf4a7cb260b564a94a1191: Beginning new tablet copy session on tablet ae6d597fbc9242539edee8685a015d99 from peer f014c4ad4c1342219a1004181ac33ed8 at {username='slave'} at 127.3.139.72:46325: session id = f014c4ad4c1342219a1004181ac33ed8-ae6d597fbc9242539edee8685a015d99
I20240502 07:59:50.617323 23625 consensus_queue.cc:237] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } }
I20240502 07:59:50.617974 20295 tablet_copy_source_session.cc:205] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:50.618836 23991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae6d597fbc9242539edee8685a015d99. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:50.619226 20933 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:50.619293 20275 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:50.619361 20406 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:50.619624 20668 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:50.619609 23625 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:50.620054 20802 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:50.620363 20142 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:50.620430 23999 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:50.620941 24001 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:50.621124 23999 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:50.621356 23991 tablet_copy_client.cc:783] T ae6d597fbc9242539edee8685a015d99 P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:50.621505 23991 tablet_copy_client.cc:647] T ae6d597fbc9242539edee8685a015d99 P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:50.621529 24000 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:50.621770 24001 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:50.623539 22741 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:50.623939 23999 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:50.624444 23625 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:50.625221 23991 tablet_copy_client.cc:523] T ae6d597fbc9242539edee8685a015d99 P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:50.625108 20802 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:50.626325 23991 tablet_bootstrap.cc:492] T ae6d597fbc9242539edee8685a015d99 P f014c4ad4c1342219a1004181ac33ed8: Bootstrap starting.
I20240502 07:59:50.626540 20142 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:50.627496 20933 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:50.628831 20275 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:50.629230 19995 catalog_manager.cc:5484] T a973de06330c487ba54a8a36b1e31b54 P 29a7180fe4574c09a8dce7c27500aab2 reported cstate change: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e951f009258e473cb57cf8773c7975d4" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:50.630388 20668 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:50.633282 22741 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:50.635936 20406 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:50.639508 23897 raft_consensus.cc:1050] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2: attempting to promote NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 to VOTER
I20240502 07:59:50.640053 23897 consensus_queue.cc:237] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } }
I20240502 07:59:50.641934 20933 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:50.641934 20802 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:50.642242 20537 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:50.642448 20668 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:50.642764 20275 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:50.642805 23897 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:50.643159 23897 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:50.643316 20406 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:50.643724 23592 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:50.644032 23592 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:50.645364 23898 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:50.645689 23898 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:50.646838 23898 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.647240 22609 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:50.647776 23592 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:50.648130 20406 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.648856 20668 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.648934 20537 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.650291 20275 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.652009 22609 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.655193 23991 tablet_bootstrap.cc:492] T ae6d597fbc9242539edee8685a015d99 P f014c4ad4c1342219a1004181ac33ed8: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:50.655093 20802 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.655422 23991 tablet_bootstrap.cc:492] T ae6d597fbc9242539edee8685a015d99 P f014c4ad4c1342219a1004181ac33ed8: Bootstrap complete.
I20240502 07:59:50.655572 23991 ts_tablet_manager.cc:1367] T ae6d597fbc9242539edee8685a015d99 P f014c4ad4c1342219a1004181ac33ed8: Time spent bootstrapping tablet: real 0.029s user 0.020s sys 0.003s
I20240502 07:59:50.655917 23991 raft_consensus.cc:348] T ae6d597fbc9242539edee8685a015d99 P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } }
I20240502 07:59:50.656003 23991 raft_consensus.cc:729] T ae6d597fbc9242539edee8685a015d99 P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f014c4ad4c1342219a1004181ac33ed8, State: Initialized, Role: LEARNER
I20240502 07:59:50.656077 23991 consensus_queue.cc:260] T ae6d597fbc9242539edee8685a015d99 P f014c4ad4c1342219a1004181ac33ed8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } }
I20240502 07:59:50.655954 20000 catalog_manager.cc:5484] T 880876d792554606a7ccf21f9ec08e0f P 35642aa2099c40e59bc2092604078e07 reported cstate change: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "29a7180fe4574c09a8dce7c27500aab2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.656330 23991 ts_tablet_manager.cc:1398] T ae6d597fbc9242539edee8685a015d99 P f014c4ad4c1342219a1004181ac33ed8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:50.657274 20295 tablet_copy_service.cc:342] P b7604c0eb3bf4a7cb260b564a94a1191: Request end of tablet copy session f014c4ad4c1342219a1004181ac33ed8-ae6d597fbc9242539edee8685a015d99 received from {username='slave'} at 127.3.139.72:46325
I20240502 07:59:50.657339 20295 tablet_copy_service.cc:433] P b7604c0eb3bf4a7cb260b564a94a1191: ending tablet copy session f014c4ad4c1342219a1004181ac33ed8-ae6d597fbc9242539edee8685a015d99 on tablet ae6d597fbc9242539edee8685a015d99 with peer f014c4ad4c1342219a1004181ac33ed8
I20240502 07:59:50.659582 20933 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.664042 20537 consensus_queue.cc:237] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } }
I20240502 07:59:50.665347 22741 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:50.665398 20406 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:50.665460 20802 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:50.665822 20933 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:50.665822 23998 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:50.665987 24000 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:50.666200 20275 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:50.666451 20142 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:50.666507 23624 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:50.666797 23624 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:50.667079 23998 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:50.668269 24000 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:50.669126 24000 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:50.670945 20405 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:50.671030 20275 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:50.671216 22741 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:50.672410 20933 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:50.673738 19993 catalog_manager.cc:5484] T a973de06330c487ba54a8a36b1e31b54 P 35642aa2099c40e59bc2092604078e07 reported cstate change: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New cstate: current_term: 1 leader_uuid: "e951f009258e473cb57cf8773c7975d4" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:50.674345 19986 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet a973de06330c487ba54a8a36b1e31b54 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:50.674983 20142 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:50.675951 20649 tablet_service.cc:1514] Processing DeleteTablet for tablet a973de06330c487ba54a8a36b1e31b54 with delete_type TABLET_DATA_TOMBSTONED (TS b443e68b6212450286869124684bdfff not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:53090
I20240502 07:59:50.677089 24026 tablet_replica.cc:330] T a973de06330c487ba54a8a36b1e31b54 P b443e68b6212450286869124684bdfff: stopping tablet replica
I20240502 07:59:50.677239 24026 raft_consensus.cc:2229] T a973de06330c487ba54a8a36b1e31b54 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:50.677424 24026 raft_consensus.cc:2258] T a973de06330c487ba54a8a36b1e31b54 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:50.678107 24026 ts_tablet_manager.cc:1875] T a973de06330c487ba54a8a36b1e31b54 P b443e68b6212450286869124684bdfff: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:50.680464 20802 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:50.680804 24026 ts_tablet_manager.cc:1888] T a973de06330c487ba54a8a36b1e31b54 P b443e68b6212450286869124684bdfff: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 07:59:50.680922 24026 log.cc:1196] T a973de06330c487ba54a8a36b1e31b54 P b443e68b6212450286869124684bdfff: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/wal/wals/a973de06330c487ba54a8a36b1e31b54
I20240502 07:59:50.681059 23770 raft_consensus.cc:1050] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff: attempting to promote NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 to VOTER
I20240502 07:59:50.681495 19987 catalog_manager.cc:4830] TS b443e68b6212450286869124684bdfff (127.3.139.69:37923): tablet a973de06330c487ba54a8a36b1e31b54 (table rebalance_test_table_0 [id=89c0cb0099e94f1aa93c3eb299e446f0]) successfully deleted
I20240502 07:59:50.681520 23770 consensus_queue.cc:237] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } }
I20240502 07:59:50.683182 22609 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:50.683337 20275 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:50.683758 23755 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:50.683821 20142 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:50.683905 20802 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:50.684082 23755 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:50.684159 20406 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:50.684258 23099 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:50.684579 23099 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:50.684635 20537 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:50.684886 23099 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:50.685129 24028 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:50.685803 20933 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:50.686164 23755 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:50.689145 24028 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.689548 20933 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.690680 22609 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.690793 20275 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.692831 20142 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.693423 19994 catalog_manager.cc:5484] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff reported cstate change: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b443e68b6212450286869124684bdfff" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:50.693734 20537 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.695169 20800 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.700502 20406 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.702168 20668 consensus_queue.cc:237] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } }
I20240502 07:59:50.703563 20272 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:50.703627 20406 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:50.703913 20142 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:50.703902 24031 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:50.704010 23099 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:50.704246 23099 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:50.704499 20933 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:50.704939 23755 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:50.705020 20537 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:50.705924 22609 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:50.705932 24029 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:50.706251 24031 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:50.706699 24031 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.707028 20933 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.707984 20537 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.708109 22609 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.708565 19987 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet bbae35bae8c047a6982e128ffb77ca86 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:50.709750 19992 catalog_manager.cc:5484] T bbae35bae8c047a6982e128ffb77ca86 P b5f471f8b5944986a7d878cf6ac550b8 reported cstate change: config changed from index 6 to 7, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New cstate: current_term: 1 leader_uuid: "b443e68b6212450286869124684bdfff" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.709653 20406 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.710601 20272 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.711959 20781 tablet_service.cc:1514] Processing DeleteTablet for tablet bbae35bae8c047a6982e128ffb77ca86 with delete_type TABLET_DATA_TOMBSTONED (TS c15e23a34059456788f9775031e27ec0 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:45388
I20240502 07:59:50.712291 20142 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.714053 24040 tablet_replica.cc:330] T bbae35bae8c047a6982e128ffb77ca86 P c15e23a34059456788f9775031e27ec0: stopping tablet replica
I20240502 07:59:50.714221 24040 raft_consensus.cc:2229] T bbae35bae8c047a6982e128ffb77ca86 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:50.714390 24040 raft_consensus.cc:2258] T bbae35bae8c047a6982e128ffb77ca86 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:50.714938 24040 ts_tablet_manager.cc:1875] T bbae35bae8c047a6982e128ffb77ca86 P c15e23a34059456788f9775031e27ec0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:50.717646 24040 ts_tablet_manager.cc:1888] T bbae35bae8c047a6982e128ffb77ca86 P c15e23a34059456788f9775031e27ec0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 07:59:50.717705 24040 log.cc:1196] T bbae35bae8c047a6982e128ffb77ca86 P c15e23a34059456788f9775031e27ec0: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/wal/wals/bbae35bae8c047a6982e128ffb77ca86
I20240502 07:59:50.718124 19986 catalog_manager.cc:4830] TS c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085): tablet bbae35bae8c047a6982e128ffb77ca86 (table rebalance_test_table_3 [id=1388635dd26f4d3791ee92dfbf8d374a]) successfully deleted
I20240502 07:59:50.768394 22609 raft_consensus.cc:1203] T 1bc5937510214c3da7731d3ad2a67842 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240502 07:59:50.771742 23562 raft_consensus.cc:1050] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8: attempting to promote NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 to VOTER
I20240502 07:59:50.772255 23562 consensus_queue.cc:237] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } }
I20240502 07:59:50.774164 20142 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:50.774250 20537 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:50.774263 20800 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:50.774474 20272 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:50.774729 24068 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:50.774775 20668 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:50.775249 24074 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:50.775405 20406 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:50.775436 24068 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:50.775791 22609 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:50.775913 24070 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:50.776050 24075 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:50.778753 24070 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:50.778919 24074 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:50.779587 24074 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.780148 20142 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.782259 20001 catalog_manager.cc:5484] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 reported cstate change: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b5f471f8b5944986a7d878cf6ac550b8" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:50.782590 20537 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.782985 20668 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.784406 20800 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.784406 20272 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.787755 20406 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.787863 22609 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.793645 20933 consensus_queue.cc:237] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } }
I20240502 07:59:50.795331 20406 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:50.795372 20142 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:50.795372 20668 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:50.795632 20800 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:50.795681 24068 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:50.795909 20272 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:50.795985 24075 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:50.796393 24075 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:50.796592 24072 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:50.796921 24072 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:50.798395 24070 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.798672 20800 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.798828 22609 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:50.798853 20668 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.800113 19988 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 15bb9480e88b41219cc91168cc438c70 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:50.800714 20405 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.800868 19994 catalog_manager.cc:5484] T 15bb9480e88b41219cc91168cc438c70 P c15e23a34059456788f9775031e27ec0 reported cstate change: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New cstate: current_term: 1 leader_uuid: "b5f471f8b5944986a7d878cf6ac550b8" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.801627 20142 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.803625 24068 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:50.804281 20272 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.804651 22609 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.808555 20517 tablet_service.cc:1514] Processing DeleteTablet for tablet 15bb9480e88b41219cc91168cc438c70 with delete_type TABLET_DATA_TOMBSTONED (TS e951f009258e473cb57cf8773c7975d4 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:43532
I20240502 07:59:50.808873 24095 tablet_replica.cc:330] T 15bb9480e88b41219cc91168cc438c70 P e951f009258e473cb57cf8773c7975d4: stopping tablet replica
I20240502 07:59:50.808982 24095 raft_consensus.cc:2229] T 15bb9480e88b41219cc91168cc438c70 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:50.809111 24095 raft_consensus.cc:2258] T 15bb9480e88b41219cc91168cc438c70 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:50.810827 22593 raft_consensus.cc:1203] T c3ca6f24c92d4273ac6c9d83161813d7 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240502 07:59:50.811331 24095 ts_tablet_manager.cc:1875] T 15bb9480e88b41219cc91168cc438c70 P e951f009258e473cb57cf8773c7975d4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:50.813593 24095 ts_tablet_manager.cc:1888] T 15bb9480e88b41219cc91168cc438c70 P e951f009258e473cb57cf8773c7975d4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 07:59:50.813659 24095 log.cc:1196] T 15bb9480e88b41219cc91168cc438c70 P e951f009258e473cb57cf8773c7975d4: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal/wals/15bb9480e88b41219cc91168cc438c70
I20240502 07:59:50.814101 19986 catalog_manager.cc:4830] TS e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315): tablet 15bb9480e88b41219cc91168cc438c70 (table rebalance_test_table_0 [id=89c0cb0099e94f1aa93c3eb299e446f0]) successfully deleted
I20240502 07:59:50.879843 23012 raft_consensus.cc:1050] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0: attempting to promote NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 to VOTER
I20240502 07:59:50.880285 23012 consensus_queue.cc:237] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } }
I20240502 07:59:50.881894 20405 raft_consensus.cc:1261] T b73e0ce533014420af513429620af68c P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:50.882030 20142 raft_consensus.cc:1261] T b73e0ce533014420af513429620af68c P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:50.882212 20933 raft_consensus.cc:1261] T b73e0ce533014420af513429620af68c P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:50.882388 23934 consensus_queue.cc:1035] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:50.882493 20668 raft_consensus.cc:1261] T b73e0ce533014420af513429620af68c P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:50.882606 23012 consensus_queue.cc:1035] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:50.882658 20537 raft_consensus.cc:1261] T b73e0ce533014420af513429620af68c P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:50.882736 23934 consensus_queue.cc:1035] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:50.883006 24100 consensus_queue.cc:1035] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:50.883291 20272 raft_consensus.cc:1261] T b73e0ce533014420af513429620af68c P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:50.883613 22609 raft_consensus.cc:1261] T b73e0ce533014420af513429620af68c P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:50.884518 23934 consensus_queue.cc:1035] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:50.884690 24098 consensus_queue.cc:1035] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:50.885041 24100 consensus_queue.cc:1035] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:50.886858 24100 raft_consensus.cc:2932] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.887344 20142 raft_consensus.cc:2932] T b73e0ce533014420af513429620af68c P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.887351 22609 raft_consensus.cc:2932] T b73e0ce533014420af513429620af68c P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.888216 20537 raft_consensus.cc:2932] T b73e0ce533014420af513429620af68c P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.889438 20933 raft_consensus.cc:2932] T b73e0ce533014420af513429620af68c P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.889952 19999 catalog_manager.cc:5484] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 reported cstate change: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c15e23a34059456788f9775031e27ec0" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:50.890195 20405 raft_consensus.cc:2932] T b73e0ce533014420af513429620af68c P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.892323 20668 raft_consensus.cc:2932] T b73e0ce533014420af513429620af68c P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.893086 20272 raft_consensus.cc:2932] T b73e0ce533014420af513429620af68c P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.897265 20800 consensus_queue.cc:237] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } }
I20240502 07:59:50.898571 20142 raft_consensus.cc:1261] T b73e0ce533014420af513429620af68c P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:50.898597 20405 raft_consensus.cc:1261] T b73e0ce533014420af513429620af68c P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:50.898625 20933 raft_consensus.cc:1261] T b73e0ce533014420af513429620af68c P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:50.898941 20272 raft_consensus.cc:1261] T b73e0ce533014420af513429620af68c P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:50.898932 23934 consensus_queue.cc:1035] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:50.899117 24108 consensus_queue.cc:1035] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:50.899293 23012 consensus_queue.cc:1035] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:50.899457 24097 consensus_queue.cc:1035] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:50.899667 20668 raft_consensus.cc:1261] T b73e0ce533014420af513429620af68c P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:50.900220 24097 consensus_queue.cc:1035] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:50.901669 23012 raft_consensus.cc:2932] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.902350 22609 raft_consensus.cc:1261] T b73e0ce533014420af513429620af68c P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:50.902222 20272 raft_consensus.cc:2932] T b73e0ce533014420af513429620af68c P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.902714 24098 consensus_queue.cc:1035] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:50.903815 20405 raft_consensus.cc:2932] T b73e0ce533014420af513429620af68c P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.904165 20668 raft_consensus.cc:2932] T b73e0ce533014420af513429620af68c P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.904263 19994 catalog_manager.cc:5484] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 reported cstate change: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New cstate: current_term: 1 leader_uuid: "c15e23a34059456788f9775031e27ec0" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:50.904372 20933 raft_consensus.cc:2932] T b73e0ce533014420af513429620af68c P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.905004 19986 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet b73e0ce533014420af513429620af68c with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:50.906836 22609 raft_consensus.cc:2932] T b73e0ce533014420af513429620af68c P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.907433 20142 raft_consensus.cc:2932] T b73e0ce533014420af513429620af68c P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:50.909291 20517 tablet_service.cc:1514] Processing DeleteTablet for tablet b73e0ce533014420af513429620af68c with delete_type TABLET_DATA_TOMBSTONED (TS e951f009258e473cb57cf8773c7975d4 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:43532
I20240502 07:59:50.909546 24095 tablet_replica.cc:330] T b73e0ce533014420af513429620af68c P e951f009258e473cb57cf8773c7975d4: stopping tablet replica
I20240502 07:59:50.909672 24095 raft_consensus.cc:2229] T b73e0ce533014420af513429620af68c P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:50.909848 24095 raft_consensus.cc:2258] T b73e0ce533014420af513429620af68c P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:50.910615 24095 ts_tablet_manager.cc:1875] T b73e0ce533014420af513429620af68c P e951f009258e473cb57cf8773c7975d4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:50.912247 24095 ts_tablet_manager.cc:1888] T b73e0ce533014420af513429620af68c P e951f009258e473cb57cf8773c7975d4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 07:59:50.912294 24095 log.cc:1196] T b73e0ce533014420af513429620af68c P e951f009258e473cb57cf8773c7975d4: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal/wals/b73e0ce533014420af513429620af68c
I20240502 07:59:50.912662 19986 catalog_manager.cc:4830] TS e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315): tablet b73e0ce533014420af513429620af68c (table rebalance_test_table_4 [id=8edc4db66dcd46ee8429441e0ce671a3]) successfully deleted
I20240502 07:59:51.019755 20537 consensus_queue.cc:237] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } }
I20240502 07:59:51.021420 20800 raft_consensus.cc:1261] T 4ce741ca5e264ccfbdde163add361886 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:51.021438 20272 raft_consensus.cc:1261] T 4ce741ca5e264ccfbdde163add361886 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:51.021438 20933 raft_consensus.cc:1261] T 4ce741ca5e264ccfbdde163add361886 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:51.021653 20142 raft_consensus.cc:1261] T 4ce741ca5e264ccfbdde163add361886 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:51.021766 20405 raft_consensus.cc:1261] T 4ce741ca5e264ccfbdde163add361886 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:51.022092 23999 consensus_queue.cc:1035] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:51.022265 23625 consensus_queue.cc:1035] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:51.022377 23624 consensus_queue.cc:1035] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:51.022644 23624 consensus_queue.cc:1035] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:51.023267 23998 consensus_queue.cc:1035] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:51.024657 20668 raft_consensus.cc:1261] T 4ce741ca5e264ccfbdde163add361886 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:51.025043 23625 consensus_queue.cc:1035] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:51.025718 23625 raft_consensus.cc:2932] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:51.026322 20933 raft_consensus.cc:2932] T 4ce741ca5e264ccfbdde163add361886 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:51.028048 20142 raft_consensus.cc:2932] T 4ce741ca5e264ccfbdde163add361886 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:51.028311 19997 catalog_manager.cc:5484] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "e951f009258e473cb57cf8773c7975d4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: HEALTHY } } }
I20240502 07:59:51.029114 20272 raft_consensus.cc:2932] T 4ce741ca5e264ccfbdde163add361886 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:51.029114 20668 raft_consensus.cc:2932] T 4ce741ca5e264ccfbdde163add361886 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:51.029448 20405 raft_consensus.cc:2932] T 4ce741ca5e264ccfbdde163add361886 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:51.031147 20800 raft_consensus.cc:2932] T 4ce741ca5e264ccfbdde163add361886 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } }
I20240502 07:59:51.033823 20537 consensus_queue.cc:237] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } }
I20240502 07:59:51.035938 20802 raft_consensus.cc:1261] T 4ce741ca5e264ccfbdde163add361886 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:51.036300 23999 consensus_queue.cc:1035] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:51.037272 20668 raft_consensus.cc:1261] T 4ce741ca5e264ccfbdde163add361886 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:51.037703 23998 consensus_queue.cc:1035] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:51.037815 20272 raft_consensus.cc:1261] T 4ce741ca5e264ccfbdde163add361886 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:51.038290 23998 consensus_queue.cc:1035] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:51.038436 20142 raft_consensus.cc:1261] T 4ce741ca5e264ccfbdde163add361886 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:51.038841 23998 consensus_queue.cc:1035] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:51.040056 20933 raft_consensus.cc:1261] T 4ce741ca5e264ccfbdde163add361886 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:51.040463 23998 consensus_queue.cc:1035] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:51.040709 20405 raft_consensus.cc:1261] T 4ce741ca5e264ccfbdde163add361886 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:51.041138 23998 consensus_queue.cc:1035] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:51.043604 23625 raft_consensus.cc:2932] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:51.044893 20668 raft_consensus.cc:2932] T 4ce741ca5e264ccfbdde163add361886 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:51.045990 20001 catalog_manager.cc:5484] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 reported cstate change: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New cstate: current_term: 1 leader_uuid: "e951f009258e473cb57cf8773c7975d4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20240502 07:59:51.046898 20472 consensus_peers.cc:485] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 -> Peer bfb1b3d596ef474faa229a835329cf20 (127.3.139.73:38741): Couldn't send request to peer bfb1b3d596ef474faa229a835329cf20. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4ce741ca5e264ccfbdde163add361886. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:51.047391 20142 raft_consensus.cc:2932] T 4ce741ca5e264ccfbdde163add361886 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:51.048938 19986 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 4ce741ca5e264ccfbdde163add361886 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240502 07:59:51.048862 20405 raft_consensus.cc:2932] T 4ce741ca5e264ccfbdde163add361886 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:51.049562 20272 raft_consensus.cc:2932] T 4ce741ca5e264ccfbdde163add361886 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:51.051992 20933 raft_consensus.cc:2932] T 4ce741ca5e264ccfbdde163add361886 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:51.052697 20802 raft_consensus.cc:2932] T 4ce741ca5e264ccfbdde163add361886 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:51.084728 22741 raft_consensus.cc:1203] T ae6d597fbc9242539edee8685a015d99 P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240502 07:59:51.235184 20142 consensus_queue.cc:237] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } }
I20240502 07:59:51.236570 22609 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:51.236809 20933 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:51.236826 20802 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:51.237114 24017 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:51.237299 24160 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:51.237421 20272 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:51.237591 24165 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:51.237689 20537 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:51.237821 24164 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:51.238219 20405 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:51.238674 24162 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:51.238853 24164 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:51.240478 24160 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:51.241138 22609 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:51.242383 20933 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:51.243117 19997 catalog_manager.cc:5484] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 reported cstate change: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New cstate: current_term: 1 leader_uuid: "29a7180fe4574c09a8dce7c27500aab2" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:51.244411 19986 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 880876d792554606a7ccf21f9ec08e0f with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:51.245216 20802 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:51.245656 20537 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:51.245752 20272 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:51.250047 20405 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:51.251227 20649 tablet_service.cc:1514] Processing DeleteTablet for tablet 880876d792554606a7ccf21f9ec08e0f with delete_type TABLET_DATA_TOMBSTONED (TS b443e68b6212450286869124684bdfff not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:53090
I20240502 07:59:51.252487 24166 tablet_replica.cc:330] T 880876d792554606a7ccf21f9ec08e0f P b443e68b6212450286869124684bdfff: stopping tablet replica
I20240502 07:59:51.252650 24166 raft_consensus.cc:2229] T 880876d792554606a7ccf21f9ec08e0f P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:51.252849 24166 raft_consensus.cc:2258] T 880876d792554606a7ccf21f9ec08e0f P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:51.253479 24166 ts_tablet_manager.cc:1875] T 880876d792554606a7ccf21f9ec08e0f P b443e68b6212450286869124684bdfff: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:51.255607 24166 ts_tablet_manager.cc:1888] T 880876d792554606a7ccf21f9ec08e0f P b443e68b6212450286869124684bdfff: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 07:59:51.255682 24166 log.cc:1196] T 880876d792554606a7ccf21f9ec08e0f P b443e68b6212450286869124684bdfff: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/wal/wals/880876d792554606a7ccf21f9ec08e0f
I20240502 07:59:51.256121 19987 catalog_manager.cc:4830] TS b443e68b6212450286869124684bdfff (127.3.139.69:37923): tablet 880876d792554606a7ccf21f9ec08e0f (table rebalance_test_table_2 [id=30a30b02a2404507a95fc3b5bb77bdd1]) successfully deleted
I20240502 07:59:51.264381 24030 raft_consensus.cc:1050] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff: attempting to promote NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 to VOTER
I20240502 07:59:51.264830 24030 consensus_queue.cc:237] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } }
I20240502 07:59:51.266553 20142 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:51.266574 20802 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:51.266682 20537 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:51.267043 24171 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:51.267318 20272 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:51.267449 20933 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:51.267513 20405 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:51.268322 24030 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:51.268671 22609 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:51.268661 23099 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:51.268960 24029 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:51.269088 23099 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:51.269227 24170 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:51.271133 24171 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:51.271994 24170 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:51.272373 20405 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:51.272938 20142 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:51.273851 20272 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:51.275144 20802 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:51.275326 19999 catalog_manager.cc:5484] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff reported cstate change: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b443e68b6212450286869124684bdfff" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:51.276356 20933 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:51.276799 20537 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:51.280529 22609 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:51.282145 20668 consensus_queue.cc:237] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } }
I20240502 07:59:51.284364 20272 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:51.284588 20802 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:51.284745 23099 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:51.284847 20405 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:51.284937 24030 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:51.285194 23099 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:51.285310 20142 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:51.285640 23099 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:51.286116 22609 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:51.286263 20933 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:51.286546 24030 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:51.286690 24029 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:51.287448 24030 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:51.288762 19987 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 1bc5937510214c3da7731d3ad2a67842 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:51.288630 20933 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:51.289631 20142 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:51.289749 20802 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:51.290863 20405 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:51.291987 19992 catalog_manager.cc:5484] T 1bc5937510214c3da7731d3ad2a67842 P 29a7180fe4574c09a8dce7c27500aab2 reported cstate change: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New cstate: current_term: 1 leader_uuid: "b443e68b6212450286869124684bdfff" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:51.294394 20517 tablet_service.cc:1514] Processing DeleteTablet for tablet 1bc5937510214c3da7731d3ad2a67842 with delete_type TABLET_DATA_TOMBSTONED (TS e951f009258e473cb57cf8773c7975d4 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:43532
I20240502 07:59:51.294672 24095 tablet_replica.cc:330] T 1bc5937510214c3da7731d3ad2a67842 P e951f009258e473cb57cf8773c7975d4: stopping tablet replica
I20240502 07:59:51.294817 24095 raft_consensus.cc:2229] T 1bc5937510214c3da7731d3ad2a67842 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:51.294991 24095 raft_consensus.cc:2258] T 1bc5937510214c3da7731d3ad2a67842 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:51.295967 24095 ts_tablet_manager.cc:1875] T 1bc5937510214c3da7731d3ad2a67842 P e951f009258e473cb57cf8773c7975d4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:51.297576 24095 ts_tablet_manager.cc:1888] T 1bc5937510214c3da7731d3ad2a67842 P e951f009258e473cb57cf8773c7975d4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 07:59:51.297652 24095 log.cc:1196] T 1bc5937510214c3da7731d3ad2a67842 P e951f009258e473cb57cf8773c7975d4: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-3/wal/wals/1bc5937510214c3da7731d3ad2a67842
I20240502 07:59:51.298158 19986 catalog_manager.cc:4830] TS e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315): tablet 1bc5937510214c3da7731d3ad2a67842 (table rebalance_test_table_1 [id=dd634821c68c405c9e104a7053619a61]) successfully deleted
I20240502 07:59:51.299590 20275 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:51.301800 22609 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:51.386392 24171 raft_consensus.cc:1050] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff: attempting to promote NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 to VOTER
I20240502 07:59:51.386968 24171 consensus_queue.cc:237] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } }
I20240502 07:59:51.388720 22609 raft_consensus.cc:1261] T c3ca6f24c92d4273ac6c9d83161813d7 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:51.388890 20933 raft_consensus.cc:1261] T c3ca6f24c92d4273ac6c9d83161813d7 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:51.388967 20537 raft_consensus.cc:1261] T c3ca6f24c92d4273ac6c9d83161813d7 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:51.389432 24170 consensus_queue.cc:1035] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:51.389662 24168 consensus_queue.cc:1035] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:51.390172 24168 consensus_queue.cc:1035] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:51.392199 20405 raft_consensus.cc:1261] T c3ca6f24c92d4273ac6c9d83161813d7 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:51.392305 20802 raft_consensus.cc:1261] T c3ca6f24c92d4273ac6c9d83161813d7 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:51.392436 20142 raft_consensus.cc:1261] T c3ca6f24c92d4273ac6c9d83161813d7 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:51.392608 24169 consensus_queue.cc:1035] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:51.392779 23099 consensus_queue.cc:1035] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:51.393159 24170 consensus_queue.cc:1035] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:51.393229 20275 raft_consensus.cc:1261] T c3ca6f24c92d4273ac6c9d83161813d7 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:51.394124 24170 consensus_queue.cc:1035] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:51.394919 24169 raft_consensus.cc:2932] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:51.395331 20802 raft_consensus.cc:2932] T c3ca6f24c92d4273ac6c9d83161813d7 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:51.395625 20405 raft_consensus.cc:2932] T c3ca6f24c92d4273ac6c9d83161813d7 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:51.396302 20142 raft_consensus.cc:2932] T c3ca6f24c92d4273ac6c9d83161813d7 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:51.398744 19994 catalog_manager.cc:5484] T c3ca6f24c92d4273ac6c9d83161813d7 P 35642aa2099c40e59bc2092604078e07 reported cstate change: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b443e68b6212450286869124684bdfff" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:51.399214 20933 raft_consensus.cc:2932] T c3ca6f24c92d4273ac6c9d83161813d7 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:51.400740 20537 raft_consensus.cc:2932] T c3ca6f24c92d4273ac6c9d83161813d7 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:51.401185 20275 raft_consensus.cc:2932] T c3ca6f24c92d4273ac6c9d83161813d7 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:51.405069 22609 raft_consensus.cc:2932] T c3ca6f24c92d4273ac6c9d83161813d7 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:51.406992 20668 consensus_queue.cc:237] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } }
I20240502 07:59:51.408646 20537 raft_consensus.cc:1261] T c3ca6f24c92d4273ac6c9d83161813d7 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:51.408648 20405 raft_consensus.cc:1261] T c3ca6f24c92d4273ac6c9d83161813d7 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:51.409029 24171 consensus_queue.cc:1035] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:51.409076 20275 raft_consensus.cc:1261] T c3ca6f24c92d4273ac6c9d83161813d7 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:51.409178 20933 raft_consensus.cc:1261] T c3ca6f24c92d4273ac6c9d83161813d7 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:51.409396 24168 consensus_queue.cc:1035] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:51.409648 23099 consensus_queue.cc:1035] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:51.410058 24170 consensus_queue.cc:1035] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:51.412235 22609 raft_consensus.cc:1261] T c3ca6f24c92d4273ac6c9d83161813d7 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:51.412295 20142 raft_consensus.cc:1261] T c3ca6f24c92d4273ac6c9d83161813d7 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:51.412601 24170 consensus_queue.cc:1035] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:51.413005 23099 consensus_queue.cc:1035] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:51.413501 24170 raft_consensus.cc:2932] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:51.414921 20537 raft_consensus.cc:2932] T c3ca6f24c92d4273ac6c9d83161813d7 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:51.415374 19987 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet c3ca6f24c92d4273ac6c9d83161813d7 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:51.416080 19994 catalog_manager.cc:5484] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff reported cstate change: config changed from index 6 to 7, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New cstate: current_term: 1 leader_uuid: "b443e68b6212450286869124684bdfff" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:51.416117 20406 raft_consensus.cc:2932] T c3ca6f24c92d4273ac6c9d83161813d7 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:51.416925 20275 raft_consensus.cc:2932] T c3ca6f24c92d4273ac6c9d83161813d7 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:51.418285 20142 raft_consensus.cc:2932] T c3ca6f24c92d4273ac6c9d83161813d7 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:51.419019 20781 tablet_service.cc:1514] Processing DeleteTablet for tablet c3ca6f24c92d4273ac6c9d83161813d7 with delete_type TABLET_DATA_TOMBSTONED (TS c15e23a34059456788f9775031e27ec0 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:45388
I20240502 07:59:51.420869 20933 raft_consensus.cc:2932] T c3ca6f24c92d4273ac6c9d83161813d7 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:51.422858 22609 raft_consensus.cc:2932] T c3ca6f24c92d4273ac6c9d83161813d7 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:51.424621 24213 tablet_replica.cc:330] T c3ca6f24c92d4273ac6c9d83161813d7 P c15e23a34059456788f9775031e27ec0: stopping tablet replica
I20240502 07:59:51.425069 24213 raft_consensus.cc:2229] T c3ca6f24c92d4273ac6c9d83161813d7 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:51.425276 24213 raft_consensus.cc:2258] T c3ca6f24c92d4273ac6c9d83161813d7 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:51.426399 24213 ts_tablet_manager.cc:1875] T c3ca6f24c92d4273ac6c9d83161813d7 P c15e23a34059456788f9775031e27ec0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:51.428176 24213 ts_tablet_manager.cc:1888] T c3ca6f24c92d4273ac6c9d83161813d7 P c15e23a34059456788f9775031e27ec0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 07:59:51.428239 24213 log.cc:1196] T c3ca6f24c92d4273ac6c9d83161813d7 P c15e23a34059456788f9775031e27ec0: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/wal/wals/c3ca6f24c92d4273ac6c9d83161813d7
I20240502 07:59:51.428674 19986 catalog_manager.cc:4830] TS c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085): tablet c3ca6f24c92d4273ac6c9d83161813d7 (table rebalance_test_table_3 [id=1388635dd26f4d3791ee92dfbf8d374a]) successfully deleted
I20240502 07:59:51.524861 24219 ts_tablet_manager.cc:902] T 4ce741ca5e264ccfbdde163add361886 P bfb1b3d596ef474faa229a835329cf20: Initiating tablet copy from peer e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315)
I20240502 07:59:51.525357 24219 tablet_copy_client.cc:308] T 4ce741ca5e264ccfbdde163add361886 P bfb1b3d596ef474faa229a835329cf20: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.68:40315
I20240502 07:59:51.529338 20537 consensus_queue.cc:237] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:51.529609 20559 tablet_copy_service.cc:140] P e951f009258e473cb57cf8773c7975d4: Received BeginTabletCopySession request for tablet 4ce741ca5e264ccfbdde163add361886 from peer bfb1b3d596ef474faa229a835329cf20 ({username='slave'} at 127.3.139.73:48527)
I20240502 07:59:51.529698 20559 tablet_copy_service.cc:161] P e951f009258e473cb57cf8773c7975d4: Beginning new tablet copy session on tablet 4ce741ca5e264ccfbdde163add361886 from peer bfb1b3d596ef474faa229a835329cf20 at {username='slave'} at 127.3.139.73:48527: session id = bfb1b3d596ef474faa229a835329cf20-4ce741ca5e264ccfbdde163add361886
I20240502 07:59:51.530635 20559 tablet_copy_source_session.cc:205] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:51.531318 20933 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:51.531343 20275 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:51.531358 20668 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:51.531430 20405 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:51.531749 23623 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:51.531939 20142 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:51.532119 24219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ce741ca5e264ccfbdde163add361886. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:51.532255 23623 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:51.532546 24001 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:51.532697 23999 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:51.532801 20802 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:51.533504 24000 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:51.533826 24000 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:51.534476 24219 tablet_copy_client.cc:783] T 4ce741ca5e264ccfbdde163add361886 P bfb1b3d596ef474faa229a835329cf20: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:51.534654 24219 tablet_copy_client.cc:647] T 4ce741ca5e264ccfbdde163add361886 P bfb1b3d596ef474faa229a835329cf20: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:51.536170 24219 tablet_copy_client.cc:523] T 4ce741ca5e264ccfbdde163add361886 P bfb1b3d596ef474faa229a835329cf20: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:51.536955 24154 raft_consensus.cc:1050] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191: attempting to promote NON_VOTER f014c4ad4c1342219a1004181ac33ed8 to VOTER
I20240502 07:59:51.536811 24001 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:51.537336 24219 tablet_bootstrap.cc:492] T 4ce741ca5e264ccfbdde163add361886 P bfb1b3d596ef474faa229a835329cf20: Bootstrap starting.
I20240502 07:59:51.538048 20802 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:51.538636 20668 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:51.538769 20405 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:51.539359 20933 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:51.541234 19993 catalog_manager.cc:5484] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "e951f009258e473cb57cf8773c7975d4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: HEALTHY } } }
I20240502 07:59:51.541656 20142 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:51.544420 20275 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:51.545282 20537 consensus_queue.cc:237] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } }
I20240502 07:59:51.547268 20405 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:51.547642 20668 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:51.548280 20935 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:51.548529 20800 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:51.548619 23625 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:51.549115 24154 consensus_queue.cc:237] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } }
I20240502 07:59:51.549407 24000 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:51.549697 24000 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:51.551306 20668 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:51.551328 22741 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:51.551666 20275 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:51.551940 20143 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:51.552222 20406 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:51.552263 23844 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:51.552577 23844 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:51.552600 20800 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:51.552738 24239 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:51.553232 20935 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:51.553781 23844 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:51.554358 23990 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:51.554502 24239 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20240502 07:59:51.556453 20472 consensus_peers.cc:485] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 -> Peer bfb1b3d596ef474faa229a835329cf20 (127.3.139.73:38741): Couldn't send request to peer bfb1b3d596ef474faa229a835329cf20. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dbf4b90cac9e4ff3b2936a1fa2ca8561. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:51.556762 24002 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:51.556888 24000 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:51.559183 23998 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:51.559564 20801 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:51.561386 20275 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:51.561547 20001 catalog_manager.cc:5484] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 reported cstate change: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New cstate: current_term: 1 leader_uuid: "e951f009258e473cb57cf8773c7975d4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:51.562816 20671 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:51.563323 19986 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet dbf4b90cac9e4ff3b2936a1fa2ca8561 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240502 07:59:51.563561 20933 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:51.564111 20142 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:51.564658 20537 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:51.565287 20406 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:51.565474 24000 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:51.566037 24237 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:51.566771 20800 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:51.567603 20539 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:51.569891 20142 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } } }
I20240502 07:59:51.572742 19997 catalog_manager.cc:5484] T ae6d597fbc9242539edee8685a015d99 P e951f009258e473cb57cf8773c7975d4 reported cstate change: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:51.573189 24219 tablet_bootstrap.cc:492] T 4ce741ca5e264ccfbdde163add361886 P bfb1b3d596ef474faa229a835329cf20: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:51.566320 24235 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:51.573849 24219 tablet_bootstrap.cc:492] T 4ce741ca5e264ccfbdde163add361886 P bfb1b3d596ef474faa229a835329cf20: Bootstrap complete.
I20240502 07:59:51.574028 24219 ts_tablet_manager.cc:1367] T 4ce741ca5e264ccfbdde163add361886 P bfb1b3d596ef474faa229a835329cf20: Time spent bootstrapping tablet: real 0.037s user 0.025s sys 0.000s
I20240502 07:59:51.574458 24219 raft_consensus.cc:348] T 4ce741ca5e264ccfbdde163add361886 P bfb1b3d596ef474faa229a835329cf20 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } }
I20240502 07:59:51.574599 24219 raft_consensus.cc:729] T 4ce741ca5e264ccfbdde163add361886 P bfb1b3d596ef474faa229a835329cf20 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: bfb1b3d596ef474faa229a835329cf20, State: Initialized, Role: LEARNER
I20240502 07:59:51.574703 24219 consensus_queue.cc:260] T 4ce741ca5e264ccfbdde163add361886 P bfb1b3d596ef474faa229a835329cf20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } }
I20240502 07:59:51.575074 24219 ts_tablet_manager.cc:1398] T 4ce741ca5e264ccfbdde163add361886 P bfb1b3d596ef474faa229a835329cf20: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:51.575598 22741 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:51.575893 20559 tablet_copy_service.cc:342] P e951f009258e473cb57cf8773c7975d4: Request end of tablet copy session bfb1b3d596ef474faa229a835329cf20-4ce741ca5e264ccfbdde163add361886 received from {username='slave'} at 127.3.139.73:48527
I20240502 07:59:51.575958 20559 tablet_copy_service.cc:433] P e951f009258e473cb57cf8773c7975d4: ending tablet copy session bfb1b3d596ef474faa229a835329cf20-4ce741ca5e264ccfbdde163add361886 on tablet 4ce741ca5e264ccfbdde163add361886 with peer bfb1b3d596ef474faa229a835329cf20
I20240502 07:59:51.576074 20935 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:51.578828 20406 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
W20240502 07:59:51.581552 24236 consensus_queue.cc:1159] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "35642aa2099c40e59bc2092604078e07" responder_term: 1 status { last_received { term: 1 index: 6 } last_committed_idx: 6 last_received_current_leader { term: 1 index: 6 } }
I20240502 07:59:51.581938 20275 consensus_queue.cc:237] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } }
I20240502 07:59:51.582049 20142 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:51.583914 20537 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:51.584121 22741 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:51.584278 24238 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:51.584501 20406 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:51.584484 23990 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:51.584803 24236 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:51.584996 20800 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:51.585883 24235 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:51.586184 20935 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:51.587173 24236 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:51.587288 23990 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:51.587697 20800 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:51.588608 19987 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet ae6d597fbc9242539edee8685a015d99 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:51.589143 20935 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:51.589471 20537 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:51.590111 20001 catalog_manager.cc:5484] T ae6d597fbc9242539edee8685a015d99 P c15e23a34059456788f9775031e27ec0 reported cstate change: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New cstate: current_term: 1 leader_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:51.590214 20406 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:51.592669 20649 tablet_service.cc:1514] Processing DeleteTablet for tablet ae6d597fbc9242539edee8685a015d99 with delete_type TABLET_DATA_TOMBSTONED (TS b443e68b6212450286869124684bdfff not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:53090
I20240502 07:59:51.592862 24166 tablet_replica.cc:330] T ae6d597fbc9242539edee8685a015d99 P b443e68b6212450286869124684bdfff: stopping tablet replica
I20240502 07:59:51.592958 20142 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:51.592988 24166 raft_consensus.cc:2229] T ae6d597fbc9242539edee8685a015d99 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:51.593101 24166 pending_rounds.cc:62] T ae6d597fbc9242539edee8685a015d99 P b443e68b6212450286869124684bdfff: Trying to abort 1 pending ops.
I20240502 07:59:51.593147 24166 pending_rounds.cc:68] T ae6d597fbc9242539edee8685a015d99 P b443e68b6212450286869124684bdfff: Aborting op as it isn't in flight: id { term: 1 index: 6 } timestamp: 7023152298136047616 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "ae6d597fbc9242539edee8685a015d99" old_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } } new_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } } }
I20240502 07:59:51.593262 24166 raft_consensus.cc:2894] T ae6d597fbc9242539edee8685a015d99 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Aborting config change with OpId 1.6: Aborted: Op aborted
I20240502 07:59:51.593302 24166 raft_consensus.cc:2866] T ae6d597fbc9242539edee8685a015d99 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20240502 07:59:51.593336 24166 raft_consensus.cc:2258] T ae6d597fbc9242539edee8685a015d99 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:51.593569 24239 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:51.594271 22741 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
W20240502 07:59:51.595877 24245 consensus_queue.cc:1159] T ae6d597fbc9242539edee8685a015d99 P b443e68b6212450286869124684bdfff [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "b443e68b6212450286869124684bdfff" status { last_received { term: 1 index: 6 } last_committed_idx: 5 last_received_current_leader { term: 1 index: 6 } }
I20240502 07:59:51.596290 20142 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:51.599615 24166 ts_tablet_manager.cc:1875] T ae6d597fbc9242539edee8685a015d99 P b443e68b6212450286869124684bdfff: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:51.602622 24166 ts_tablet_manager.cc:1888] T ae6d597fbc9242539edee8685a015d99 P b443e68b6212450286869124684bdfff: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 07:59:51.602697 24166 log.cc:1196] T ae6d597fbc9242539edee8685a015d99 P b443e68b6212450286869124684bdfff: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/wal/wals/ae6d597fbc9242539edee8685a015d99
I20240502 07:59:51.603135 19987 catalog_manager.cc:4830] TS b443e68b6212450286869124684bdfff (127.3.139.69:37923): tablet ae6d597fbc9242539edee8685a015d99 (table rebalance_test_table_5 [id=ed17eb489c6e490e9427e387589e51fe]) successfully deleted
I20240502 07:59:51.823443 20539 consensus_queue.cc:237] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } }
I20240502 07:59:51.825170 20142 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:51.825170 20800 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:51.825300 20275 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:51.825300 20668 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:51.825714 23998 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:51.825918 24000 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:51.826187 23999 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:51.826244 20935 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:51.826342 20406 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:51.826591 24000 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:51.827128 23625 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:51.827431 23625 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:51.828986 23625 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:51.829424 20275 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:51.829871 20668 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:51.831600 20406 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:51.831725 20142 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:51.832643 20001 catalog_manager.cc:5484] T fb0f04b72dc046c0afee40b895e17a1d P b443e68b6212450286869124684bdfff reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "e951f009258e473cb57cf8773c7975d4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:51.836892 20539 consensus_queue.cc:237] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } }
I20240502 07:59:51.838606 20406 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:51.838922 20668 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:51.838907 20935 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:51.838933 20275 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20240502 07:59:51.839560 20474 consensus_peers.cc:485] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fb0f04b72dc046c0afee40b895e17a1d. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:51.839649 24000 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:51.839757 24002 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:51.839839 23999 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:51.839921 23624 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:51.840809 20935 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:51.841980 20800 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:51.842944 23999 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:51.843225 24000 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:51.843667 20935 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:51.844172 20800 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } }
I20240502 07:59:51.845263 19986 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet fb0f04b72dc046c0afee40b895e17a1d with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240502 07:59:51.845682 20406 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:51.846482 19994 catalog_manager.cc:5484] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 reported cstate change: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New cstate: current_term: 1 leader_uuid: "e951f009258e473cb57cf8773c7975d4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:51.846868 20142 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:51.846788 20800 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:51.847259 23999 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:51.849148 20275 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:51.849329 20142 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:51.855099 20668 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:51.959936 22475 raft_consensus.cc:1203] T 4ce741ca5e264ccfbdde163add361886 P bfb1b3d596ef474faa229a835329cf20 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240502 07:59:52.075845 24219 ts_tablet_manager.cc:902] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P bfb1b3d596ef474faa229a835329cf20: Initiating tablet copy from peer e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315)
I20240502 07:59:52.076315 24219 tablet_copy_client.cc:308] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P bfb1b3d596ef474faa229a835329cf20: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.68:40315
I20240502 07:59:52.076763 20559 tablet_copy_service.cc:140] P e951f009258e473cb57cf8773c7975d4: Received BeginTabletCopySession request for tablet dbf4b90cac9e4ff3b2936a1fa2ca8561 from peer bfb1b3d596ef474faa229a835329cf20 ({username='slave'} at 127.3.139.73:48527)
I20240502 07:59:52.076874 20559 tablet_copy_service.cc:161] P e951f009258e473cb57cf8773c7975d4: Beginning new tablet copy session on tablet dbf4b90cac9e4ff3b2936a1fa2ca8561 from peer bfb1b3d596ef474faa229a835329cf20 at {username='slave'} at 127.3.139.73:48527: session id = bfb1b3d596ef474faa229a835329cf20-dbf4b90cac9e4ff3b2936a1fa2ca8561
I20240502 07:59:52.077803 20559 tablet_copy_source_session.cc:205] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:52.078686 24219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbf4b90cac9e4ff3b2936a1fa2ca8561. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:52.080646 24219 tablet_copy_client.cc:783] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P bfb1b3d596ef474faa229a835329cf20: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:52.080801 24219 tablet_copy_client.cc:647] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P bfb1b3d596ef474faa229a835329cf20: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:52.081888 24219 tablet_copy_client.cc:523] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P bfb1b3d596ef474faa229a835329cf20: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:52.082811 24219 tablet_bootstrap.cc:492] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P bfb1b3d596ef474faa229a835329cf20: Bootstrap starting.
I20240502 07:59:52.103660 24219 tablet_bootstrap.cc:492] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P bfb1b3d596ef474faa229a835329cf20: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:52.103893 24219 tablet_bootstrap.cc:492] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P bfb1b3d596ef474faa229a835329cf20: Bootstrap complete.
I20240502 07:59:52.104070 24219 ts_tablet_manager.cc:1367] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P bfb1b3d596ef474faa229a835329cf20: Time spent bootstrapping tablet: real 0.021s user 0.021s sys 0.000s
I20240502 07:59:52.104451 24219 raft_consensus.cc:348] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P bfb1b3d596ef474faa229a835329cf20 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } }
I20240502 07:59:52.104561 24219 raft_consensus.cc:729] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P bfb1b3d596ef474faa229a835329cf20 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: bfb1b3d596ef474faa229a835329cf20, State: Initialized, Role: LEARNER
I20240502 07:59:52.104653 24219 consensus_queue.cc:260] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P bfb1b3d596ef474faa229a835329cf20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: NON_VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: true } }
I20240502 07:59:52.105296 24219 ts_tablet_manager.cc:1398] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P bfb1b3d596ef474faa229a835329cf20: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 07:59:52.106050 20559 tablet_copy_service.cc:342] P e951f009258e473cb57cf8773c7975d4: Request end of tablet copy session bfb1b3d596ef474faa229a835329cf20-dbf4b90cac9e4ff3b2936a1fa2ca8561 received from {username='slave'} at 127.3.139.73:48527
I20240502 07:59:52.106128 20559 tablet_copy_service.cc:433] P e951f009258e473cb57cf8773c7975d4: ending tablet copy session bfb1b3d596ef474faa229a835329cf20-dbf4b90cac9e4ff3b2936a1fa2ca8561 on tablet dbf4b90cac9e4ff3b2936a1fa2ca8561 with peer bfb1b3d596ef474faa229a835329cf20
I20240502 07:59:52.312505 24332 ts_tablet_manager.cc:902] T fb0f04b72dc046c0afee40b895e17a1d P f014c4ad4c1342219a1004181ac33ed8: Initiating tablet copy from peer e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315)
I20240502 07:59:52.312940 24332 tablet_copy_client.cc:308] T fb0f04b72dc046c0afee40b895e17a1d P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.68:40315
I20240502 07:59:52.313351 20559 tablet_copy_service.cc:140] P e951f009258e473cb57cf8773c7975d4: Received BeginTabletCopySession request for tablet fb0f04b72dc046c0afee40b895e17a1d from peer f014c4ad4c1342219a1004181ac33ed8 ({username='slave'} at 127.3.139.72:45019)
I20240502 07:59:52.313444 20559 tablet_copy_service.cc:161] P e951f009258e473cb57cf8773c7975d4: Beginning new tablet copy session on tablet fb0f04b72dc046c0afee40b895e17a1d from peer f014c4ad4c1342219a1004181ac33ed8 at {username='slave'} at 127.3.139.72:45019: session id = f014c4ad4c1342219a1004181ac33ed8-fb0f04b72dc046c0afee40b895e17a1d
I20240502 07:59:52.314366 20559 tablet_copy_source_session.cc:205] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:52.315317 24332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb0f04b72dc046c0afee40b895e17a1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:52.317313 24332 tablet_copy_client.cc:783] T fb0f04b72dc046c0afee40b895e17a1d P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:52.317441 24332 tablet_copy_client.cc:647] T fb0f04b72dc046c0afee40b895e17a1d P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:52.318387 24332 tablet_copy_client.cc:523] T fb0f04b72dc046c0afee40b895e17a1d P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:52.319379 24332 tablet_bootstrap.cc:492] T fb0f04b72dc046c0afee40b895e17a1d P f014c4ad4c1342219a1004181ac33ed8: Bootstrap starting.
I20240502 07:59:52.337747 24332 tablet_bootstrap.cc:492] T fb0f04b72dc046c0afee40b895e17a1d P f014c4ad4c1342219a1004181ac33ed8: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:52.337996 24332 tablet_bootstrap.cc:492] T fb0f04b72dc046c0afee40b895e17a1d P f014c4ad4c1342219a1004181ac33ed8: Bootstrap complete.
I20240502 07:59:52.338155 24332 ts_tablet_manager.cc:1367] T fb0f04b72dc046c0afee40b895e17a1d P f014c4ad4c1342219a1004181ac33ed8: Time spent bootstrapping tablet: real 0.019s user 0.012s sys 0.003s
I20240502 07:59:52.338506 24332 raft_consensus.cc:348] T fb0f04b72dc046c0afee40b895e17a1d P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } }
I20240502 07:59:52.338622 24332 raft_consensus.cc:729] T fb0f04b72dc046c0afee40b895e17a1d P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f014c4ad4c1342219a1004181ac33ed8, State: Initialized, Role: LEARNER
I20240502 07:59:52.338709 24332 consensus_queue.cc:260] T fb0f04b72dc046c0afee40b895e17a1d P f014c4ad4c1342219a1004181ac33ed8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } }
I20240502 07:59:52.339031 24332 ts_tablet_manager.cc:1398] T fb0f04b72dc046c0afee40b895e17a1d P f014c4ad4c1342219a1004181ac33ed8: Time spent starting tablet: real 0.001s user 0.003s sys 0.001s
I20240502 07:59:52.339586 20559 tablet_copy_service.cc:342] P e951f009258e473cb57cf8773c7975d4: Request end of tablet copy session f014c4ad4c1342219a1004181ac33ed8-fb0f04b72dc046c0afee40b895e17a1d received from {username='slave'} at 127.3.139.72:45019
I20240502 07:59:52.339660 20559 tablet_copy_service.cc:433] P e951f009258e473cb57cf8773c7975d4: ending tablet copy session f014c4ad4c1342219a1004181ac33ed8-fb0f04b72dc046c0afee40b895e17a1d on tablet fb0f04b72dc046c0afee40b895e17a1d with peer f014c4ad4c1342219a1004181ac33ed8
I20240502 07:59:52.351958 23624 raft_consensus.cc:1050] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4: attempting to promote NON_VOTER bfb1b3d596ef474faa229a835329cf20 to VOTER
I20240502 07:59:52.352463 23624 consensus_queue.cc:237] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } }
I20240502 07:59:52.354410 20935 raft_consensus.cc:1261] T 4ce741ca5e264ccfbdde163add361886 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:52.354461 20668 raft_consensus.cc:1261] T 4ce741ca5e264ccfbdde163add361886 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:52.354756 20801 raft_consensus.cc:1261] T 4ce741ca5e264ccfbdde163add361886 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:52.354813 20142 raft_consensus.cc:1261] T 4ce741ca5e264ccfbdde163add361886 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:52.354889 20406 raft_consensus.cc:1261] T 4ce741ca5e264ccfbdde163add361886 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:52.354987 23999 consensus_queue.cc:1035] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:52.355302 24343 consensus_queue.cc:1035] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:52.355512 23625 consensus_queue.cc:1035] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:52.355876 22475 raft_consensus.cc:1261] T 4ce741ca5e264ccfbdde163add361886 P bfb1b3d596ef474faa229a835329cf20 [term 1 LEARNER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:52.356158 20272 raft_consensus.cc:1261] T 4ce741ca5e264ccfbdde163add361886 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:52.356226 24342 consensus_queue.cc:1035] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:52.356513 24001 consensus_queue.cc:1035] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:52.356659 24342 consensus_queue.cc:1035] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:52.357012 24001 consensus_queue.cc:1035] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:52.359946 23625 raft_consensus.cc:2932] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:52.360364 20406 raft_consensus.cc:2932] T 4ce741ca5e264ccfbdde163add361886 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:52.361521 20935 raft_consensus.cc:2932] T 4ce741ca5e264ccfbdde163add361886 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:52.362751 22475 raft_consensus.cc:2932] T 4ce741ca5e264ccfbdde163add361886 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:52.362965 19997 catalog_manager.cc:5484] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 reported cstate change: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e951f009258e473cb57cf8773c7975d4" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:52.363257 20668 raft_consensus.cc:2932] T 4ce741ca5e264ccfbdde163add361886 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:52.364105 20142 raft_consensus.cc:2932] T 4ce741ca5e264ccfbdde163add361886 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:52.366135 20801 raft_consensus.cc:2932] T 4ce741ca5e264ccfbdde163add361886 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:52.369014 20272 raft_consensus.cc:2932] T 4ce741ca5e264ccfbdde163add361886 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:52.372272 20539 consensus_queue.cc:237] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } }
I20240502 07:59:52.374003 20272 raft_consensus.cc:1261] T 4ce741ca5e264ccfbdde163add361886 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:52.374302 20935 raft_consensus.cc:1261] T 4ce741ca5e264ccfbdde163add361886 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:52.374531 20801 raft_consensus.cc:1261] T 4ce741ca5e264ccfbdde163add361886 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:52.374588 22475 raft_consensus.cc:1261] T 4ce741ca5e264ccfbdde163add361886 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:52.374989 20142 raft_consensus.cc:1261] T 4ce741ca5e264ccfbdde163add361886 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:52.375151 23624 consensus_queue.cc:1035] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:52.375593 24343 consensus_queue.cc:1035] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:52.375763 20406 raft_consensus.cc:1261] T 4ce741ca5e264ccfbdde163add361886 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:52.375945 24001 consensus_queue.cc:1035] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:52.376125 24340 consensus_queue.cc:1035] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:52.377650 24001 consensus_queue.cc:1035] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:52.378084 24343 consensus_queue.cc:1035] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:52.379525 24340 raft_consensus.cc:2932] T 4ce741ca5e264ccfbdde163add361886 P e951f009258e473cb57cf8773c7975d4 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:52.380975 20935 raft_consensus.cc:2932] T 4ce741ca5e264ccfbdde163add361886 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:52.381464 20406 raft_consensus.cc:2932] T 4ce741ca5e264ccfbdde163add361886 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:52.381484 20272 raft_consensus.cc:2932] T 4ce741ca5e264ccfbdde163add361886 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:52.383291 20801 raft_consensus.cc:2932] T 4ce741ca5e264ccfbdde163add361886 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:52.383591 20142 raft_consensus.cc:2932] T 4ce741ca5e264ccfbdde163add361886 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:52.384449 19986 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 4ce741ca5e264ccfbdde163add361886 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:52.384629 22475 raft_consensus.cc:2932] T 4ce741ca5e264ccfbdde163add361886 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:52.386145 19994 catalog_manager.cc:5484] T 4ce741ca5e264ccfbdde163add361886 P c15e23a34059456788f9775031e27ec0 reported cstate change: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New cstate: current_term: 1 leader_uuid: "e951f009258e473cb57cf8773c7975d4" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:52.395632 20649 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ce741ca5e264ccfbdde163add361886 with delete_type TABLET_DATA_TOMBSTONED (TS b443e68b6212450286869124684bdfff not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:53090
I20240502 07:59:52.396085 24356 tablet_replica.cc:330] T 4ce741ca5e264ccfbdde163add361886 P b443e68b6212450286869124684bdfff: stopping tablet replica
I20240502 07:59:52.396296 24356 raft_consensus.cc:2229] T 4ce741ca5e264ccfbdde163add361886 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:52.396580 24356 raft_consensus.cc:2258] T 4ce741ca5e264ccfbdde163add361886 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:52.397292 24356 ts_tablet_manager.cc:1875] T 4ce741ca5e264ccfbdde163add361886 P b443e68b6212450286869124684bdfff: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:52.399075 24356 ts_tablet_manager.cc:1888] T 4ce741ca5e264ccfbdde163add361886 P b443e68b6212450286869124684bdfff: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 07:59:52.399143 24356 log.cc:1196] T 4ce741ca5e264ccfbdde163add361886 P b443e68b6212450286869124684bdfff: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/wal/wals/4ce741ca5e264ccfbdde163add361886
I20240502 07:59:52.399616 19987 catalog_manager.cc:4830] TS b443e68b6212450286869124684bdfff (127.3.139.69:37923): tablet 4ce741ca5e264ccfbdde163add361886 (table rebalance_test_table_1 [id=dd634821c68c405c9e104a7053619a61]) successfully deleted
I20240502 07:59:52.622293 22475 raft_consensus.cc:1203] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P bfb1b3d596ef474faa229a835329cf20 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240502 07:59:52.811858 22741 raft_consensus.cc:1203] T fb0f04b72dc046c0afee40b895e17a1d P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240502 07:59:53.007807 24340 raft_consensus.cc:1050] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4: attempting to promote NON_VOTER bfb1b3d596ef474faa229a835329cf20 to VOTER
I20240502 07:59:53.008275 24340 consensus_queue.cc:237] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } }
I20240502 07:59:53.010109 20142 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:53.010133 20272 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:53.010556 20668 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:53.010550 24396 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:53.010601 20801 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:53.010617 20935 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:53.010898 22475 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P bfb1b3d596ef474faa229a835329cf20 [term 1 LEARNER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:53.010965 24394 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:53.011263 24395 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:53.011293 20405 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:53.011413 24394 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:53.011664 24395 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:53.011806 24340 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:53.012432 23999 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:53.014919 23624 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:53.015293 20668 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:53.015421 22475 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:53.015890 20405 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:53.017807 20801 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:53.018379 19992 catalog_manager.cc:5484] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 reported cstate change: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e951f009258e473cb57cf8773c7975d4" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:53.018852 20142 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:53.019887 20272 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:53.021322 20935 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bfb1b3d596ef474faa229a835329cf20 (127.3.139.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:53.026906 20539 consensus_queue.cc:237] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } }
I20240502 07:59:53.028225 22475 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:53.028225 20272 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:53.028419 20405 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:53.028419 20142 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:53.028602 23999 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:53.028772 24395 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:53.028879 24396 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:53.029773 23999 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:53.030761 20935 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:53.031072 20801 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:53.031140 23999 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:53.031702 24396 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:53.032049 20405 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:53.033129 19986 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet dbf4b90cac9e4ff3b2936a1fa2ca8561 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:53.033113 20935 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:53.033578 23999 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:53.033682 20142 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:53.034621 20272 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:53.034721 20801 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:53.034948 20000 catalog_manager.cc:5484] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 35642aa2099c40e59bc2092604078e07 reported cstate change: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New cstate: current_term: 1 leader_uuid: "e951f009258e473cb57cf8773c7975d4" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:53.036512 22475 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } }
I20240502 07:59:53.041215 20649 tablet_service.cc:1514] Processing DeleteTablet for tablet dbf4b90cac9e4ff3b2936a1fa2ca8561 with delete_type TABLET_DATA_TOMBSTONED (TS b443e68b6212450286869124684bdfff not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:53090
I20240502 07:59:53.041872 24410 tablet_replica.cc:330] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b443e68b6212450286869124684bdfff: stopping tablet replica
I20240502 07:59:53.042160 24410 raft_consensus.cc:2229] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:53.042436 24410 raft_consensus.cc:2258] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:53.043160 24410 ts_tablet_manager.cc:1875] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b443e68b6212450286869124684bdfff: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:53.044885 24410 ts_tablet_manager.cc:1888] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b443e68b6212450286869124684bdfff: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 07:59:53.044947 24410 log.cc:1196] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b443e68b6212450286869124684bdfff: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/wal/wals/dbf4b90cac9e4ff3b2936a1fa2ca8561
I20240502 07:59:53.045384 19987 catalog_manager.cc:4830] TS b443e68b6212450286869124684bdfff (127.3.139.69:37923): tablet dbf4b90cac9e4ff3b2936a1fa2ca8561 (table rebalance_test_table_2 [id=30a30b02a2404507a95fc3b5bb77bdd1]) successfully deleted
I20240502 07:59:53.358043 24396 raft_consensus.cc:1050] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4: attempting to promote NON_VOTER f014c4ad4c1342219a1004181ac33ed8 to VOTER
I20240502 07:59:53.358618 24396 consensus_queue.cc:237] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } }
I20240502 07:59:53.360575 20801 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:53.360605 22741 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:53.361029 20406 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:53.361210 20668 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:53.361599 24394 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:53.361714 20935 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:53.361877 20272 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:53.362056 20142 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:53.362026 24394 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:53.362313 24394 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:53.362591 24395 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:53.362785 24394 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:53.363085 24395 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:53.363270 24396 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:53.365301 24394 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:53.365733 20406 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:53.368114 22741 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:53.369146 20801 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:53.369779 20142 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:53.370733 20272 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:53.371467 20668 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:53.374050 19995 catalog_manager.cc:5484] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 reported cstate change: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e951f009258e473cb57cf8773c7975d4" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:53.374382 20935 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { replace: true } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:53.379793 20539 consensus_queue.cc:237] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } }
I20240502 07:59:53.381242 22741 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:53.381517 20935 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:53.381644 20142 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:53.381750 20801 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:53.381868 20272 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:53.381879 24340 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:53.382050 24396 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:53.382174 23624 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:53.382433 24394 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:53.382915 23999 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:53.383749 20406 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:53.384112 23999 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:53.385079 24394 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:53.386308 20272 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:53.387741 19994 catalog_manager.cc:5484] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 reported cstate change: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New cstate: current_term: 1 leader_uuid: "e951f009258e473cb57cf8773c7975d4" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:53.387744 19986 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet fb0f04b72dc046c0afee40b895e17a1d with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:53.388186 20935 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:53.388795 20143 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:53.388864 20801 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:53.390733 20649 tablet_service.cc:1514] Processing DeleteTablet for tablet fb0f04b72dc046c0afee40b895e17a1d with delete_type TABLET_DATA_TOMBSTONED (TS b443e68b6212450286869124684bdfff not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:53090
I20240502 07:59:53.390931 24410 tablet_replica.cc:330] T fb0f04b72dc046c0afee40b895e17a1d P b443e68b6212450286869124684bdfff: stopping tablet replica
I20240502 07:59:53.391088 24410 raft_consensus.cc:2229] T fb0f04b72dc046c0afee40b895e17a1d P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:53.391288 24410 raft_consensus.cc:2258] T fb0f04b72dc046c0afee40b895e17a1d P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:53.392132 24410 ts_tablet_manager.cc:1875] T fb0f04b72dc046c0afee40b895e17a1d P b443e68b6212450286869124684bdfff: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:53.393158 22741 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:53.393640 20406 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER b443e68b6212450286869124684bdfff (127.3.139.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:53.394260 24410 ts_tablet_manager.cc:1888] T fb0f04b72dc046c0afee40b895e17a1d P b443e68b6212450286869124684bdfff: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 07:59:53.394317 24410 log.cc:1196] T fb0f04b72dc046c0afee40b895e17a1d P b443e68b6212450286869124684bdfff: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-4/wal/wals/fb0f04b72dc046c0afee40b895e17a1d
I20240502 07:59:53.394778 19987 catalog_manager.cc:4830] TS b443e68b6212450286869124684bdfff (127.3.139.69:37923): tablet fb0f04b72dc046c0afee40b895e17a1d (table rebalance_test_table_6 [id=db1e20e82b894565a302e53bd7791607]) successfully deleted
I20240502 07:59:53.662377 20122 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:59:53.665310 20649 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:59:53.668699 20915 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:59:53.674774 20386 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:59:53.675599 22456 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:53.683079 20253 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:59:53.683969 20517 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:59:53.694958 22588 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:53.715746 20781 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:59:53.732486 22721 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:53.822896 20915 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:59:53.825282 20386 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:59:53.825841 20122 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:59:53.830325 20253 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:59:53.838914 22721 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:53.839589 22456 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:53.848372 22588 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:53.857585 20649 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:59:53.871116 20517 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:59:53.871335 20781 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:59:53.921880 20801 consensus_queue.cc:237] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } }
I20240502 07:59:53.923815 20406 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:53.923892 20539 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:53.924257 24436 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:53.924455 24534 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:53.924835 20935 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:53.924810 20272 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:53.925402 22475 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:53.926396 20142 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:53.925987 24534 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:53.927230 24436 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:53.927505 24212 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:53.928193 24535 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20240502 07:59:53.928673 20738 consensus_peers.cc:485] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 331368e52904494fa5ecf1ef594821fc. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:53.929698 24436 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:53.930742 20142 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:53.931627 20539 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:53.932088 20272 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:53.932511 20406 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:53.935130 19999 catalog_manager.cc:5484] T 331368e52904494fa5ecf1ef594821fc P b7604c0eb3bf4a7cb260b564a94a1191 reported cstate change: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New cstate: current_term: 1 leader_uuid: "c15e23a34059456788f9775031e27ec0" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:53.936025 20935 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:53.936640 22475 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:53.944770 20539 consensus_queue.cc:237] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } }
I20240502 07:59:53.946300 20406 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:53.946300 22475 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:53.946475 20142 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:53.946662 24396 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:53.946911 20272 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:53.946938 24422 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:53.947157 24396 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:53.947343 24553 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:53.947984 20801 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:53.948782 20935 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:53.949107 23999 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:53.949230 24396 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:53.949479 24422 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:53.949860 22475 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:53.949918 20142 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:53.950414 20935 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:53.952589 20272 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:53.952847 20801 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:53.953969 20406 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:53.954193 20001 catalog_manager.cc:5484] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b5f471f8b5944986a7d878cf6ac550b8 reported cstate change: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New cstate: current_term: 1 leader_uuid: "e951f009258e473cb57cf8773c7975d4" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
W20240502 07:59:53.960145 20474 consensus_peers.cc:485] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dbf4b90cac9e4ff3b2936a1fa2ca8561. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:53.964654 20668 consensus_queue.cc:237] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } }
I20240502 07:59:53.966398 20406 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:53.966718 20935 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:53.966718 20272 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
W20240502 07:59:53.966845 20606 consensus_peers.cc:485] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1bc5937510214c3da7731d3ad2a67842. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:53.966908 20801 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:53.967113 24355 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:53.967301 24564 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:53.967444 22609 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:53.967578 20142 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:53.967586 24355 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:53.967873 24355 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:53.968153 24355 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:53.969516 24355 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:53.970803 24567 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:53.971238 20406 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:53.971338 20142 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:53.972921 20801 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:53.973613 19999 catalog_manager.cc:5484] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff reported cstate change: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New cstate: current_term: 1 leader_uuid: "b443e68b6212450286869124684bdfff" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:53.974797 20272 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:53.974871 20935 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:53.977355 22609 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:53.984701 20406 consensus_queue.cc:237] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } }
I20240502 07:59:53.986479 22475 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:53.986899 24583 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:53.987195 20272 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:53.987313 20801 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:53.987500 20142 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:53.987557 20539 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:53.987881 24582 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:53.988055 20935 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:53.988214 24583 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:53.988380 24585 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:53.988806 24582 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:53.989001 24588 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:53.991232 24585 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:53.991582 20539 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:53.993816 22475 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:53.994144 20272 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:53.994727 20801 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:53.996369 19999 catalog_manager.cc:5484] T e61fe3cb61c240f9b2c898aff1951144 P bfb1b3d596ef474faa229a835329cf20 reported cstate change: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New cstate: current_term: 1 leader_uuid: "35642aa2099c40e59bc2092604078e07" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
W20240502 07:59:53.997277 20342 consensus_peers.cc:485] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e61fe3cb61c240f9b2c898aff1951144. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:53.998723 20935 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:53.999897 20142 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:54.007566 20935 consensus_queue.cc:237] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } }
I20240502 07:59:54.009526 22475 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:54.010147 20801 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:54.010265 20142 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:54.010373 20272 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:54.010427 20405 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:54.010490 20539 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:54.010650 24075 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:54.010974 24604 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:54.011195 24075 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:54.011456 24075 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:54.011626 24604 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:54.011902 24075 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20240502 07:59:54.015425 20870 consensus_peers.cc:485] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f588a3ec49a24b1a9a9e19cdf8c30ca5. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:54.015254 24604 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:54.015771 22475 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:54.016577 20142 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:54.016602 20539 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:54.018651 19997 catalog_manager.cc:5484] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P bfb1b3d596ef474faa229a835329cf20 reported cstate change: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New cstate: current_term: 1 leader_uuid: "b5f471f8b5944986a7d878cf6ac550b8" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:54.020341 20801 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:54.020431 20272 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:54.022462 20405 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:54.030738 20935 consensus_queue.cc:237] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } }
I20240502 07:59:54.032395 22475 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:54.032469 20272 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:54.032481 20668 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:54.032698 20539 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:54.032940 24600 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:54.033010 20405 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:54.033097 24603 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:54.033278 24601 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:54.033396 24075 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20240502 07:59:54.033596 20870 consensus_peers.cc:485] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1453e833d0ad494583c44b5496088f6d. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:54.033752 24075 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:54.033855 20142 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:54.034852 24603 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:54.036170 24603 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:54.036862 20142 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:54.037185 20272 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:54.037185 20668 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:54.039568 20001 catalog_manager.cc:5484] T 1453e833d0ad494583c44b5496088f6d P 29a7180fe4574c09a8dce7c27500aab2 reported cstate change: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New cstate: current_term: 1 leader_uuid: "b5f471f8b5944986a7d878cf6ac550b8" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:54.039990 22475 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:54.041525 20405 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:54.044592 20539 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:54.048978 20272 consensus_queue.cc:237] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } }
I20240502 07:59:54.050913 22475 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:54.050989 20539 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
W20240502 07:59:54.051497 20210 consensus_peers.cc:485] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e2908de10684c84af542f16ff8727cb. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:54.051728 20935 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:54.051509 24483 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20240502 07:59:54.052412 24633 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:54.052747 20668 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:54.052772 20405 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:54.052968 24632 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:54.053440 24633 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:54.053681 24632 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:54.055439 24632 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:54.055629 20142 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:54.056017 24316 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:54.055907 20935 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:54.056303 20405 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:54.057212 20142 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:54.057590 20668 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:54.058825 20539 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:54.059844 20001 catalog_manager.cc:5484] T 0e2908de10684c84af542f16ff8727cb P b5f471f8b5944986a7d878cf6ac550b8 reported cstate change: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New cstate: current_term: 1 leader_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:54.060412 22475 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:54.070086 20142 consensus_queue.cc:237] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } }
I20240502 07:59:54.072021 22741 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
W20240502 07:59:54.072432 20078 consensus_peers.cc:485] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9ad40762dc784dfa9f3507e025db32b8. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:54.072459 20539 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:54.072588 20272 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:54.072553 24651 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:54.072968 20935 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:54.073026 24649 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:54.073068 20405 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:54.073411 24611 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:54.073736 24611 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:54.073917 20668 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:54.074687 24651 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:54.075600 24653 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:54.076841 24649 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:54.077534 20272 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:54.077991 20668 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:54.079504 20405 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:54.079704 19997 catalog_manager.cc:5484] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 reported cstate change: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New cstate: current_term: 1 leader_uuid: "29a7180fe4574c09a8dce7c27500aab2" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:54.080955 20935 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:54.082166 20539 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:54.084416 22741 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:54.092070 20935 consensus_queue.cc:237] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } }
I20240502 07:59:54.093767 20272 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:54.093784 20668 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:54.094064 20405 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:54.094058 20142 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:54.094111 22609 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:54.094204 24618 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20240502 07:59:54.094492 20870 consensus_peers.cc:485] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 15bb9480e88b41219cc91168cc438c70. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:54.094516 20801 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:54.094738 24601 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:54.094902 24600 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:54.095106 24075 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:54.095232 24618 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:54.095407 24603 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:54.097342 24601 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:54.097641 20142 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:54.099716 20668 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:54.099874 20405 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:54.100314 19992 catalog_manager.cc:5484] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 reported cstate change: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New cstate: current_term: 1 leader_uuid: "b5f471f8b5944986a7d878cf6ac550b8" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:54.104600 20801 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:54.106433 22609 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:54.106499 20272 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:54.116813 20272 consensus_queue.cc:237] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } }
I20240502 07:59:54.118955 22741 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:54.118970 20539 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:54.119117 20801 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:54.119242 20935 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:54.119585 20405 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:54.119578 24316 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:54.119882 24483 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:54.120072 24316 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:54.120237 24635 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:54.120414 24634 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:54.120648 20142 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:54.121004 24316 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:54.124420 24635 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:54.124730 20935 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:54.125908 20405 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:54.126749 20142 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:54.127610 19997 catalog_manager.cc:5484] T ae6d597fbc9242539edee8685a015d99 P b5f471f8b5944986a7d878cf6ac550b8 reported cstate change: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New cstate: current_term: 1 leader_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:54.128258 20801 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
W20240502 07:59:54.131222 20210 consensus_peers.cc:485] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ae6d597fbc9242539edee8685a015d99. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:54.131662 22741 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:54.134083 20539 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:54.424033 24767 ts_tablet_manager.cc:902] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P f014c4ad4c1342219a1004181ac33ed8: Initiating tablet copy from peer b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577)
I20240502 07:59:54.424750 24767 tablet_copy_client.cc:308] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.71:39577
I20240502 07:59:54.426501 24777 ts_tablet_manager.cc:902] T 331368e52904494fa5ecf1ef594821fc P f014c4ad4c1342219a1004181ac33ed8: Initiating tablet copy from peer c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085)
I20240502 07:59:54.426990 24777 tablet_copy_client.cc:308] T 331368e52904494fa5ecf1ef594821fc P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.70:36085
I20240502 07:59:54.430760 20955 tablet_copy_service.cc:140] P b5f471f8b5944986a7d878cf6ac550b8: Received BeginTabletCopySession request for tablet f588a3ec49a24b1a9a9e19cdf8c30ca5 from peer f014c4ad4c1342219a1004181ac33ed8 ({username='slave'} at 127.3.139.72:53857)
I20240502 07:59:54.430872 20955 tablet_copy_service.cc:161] P b5f471f8b5944986a7d878cf6ac550b8: Beginning new tablet copy session on tablet f588a3ec49a24b1a9a9e19cdf8c30ca5 from peer f014c4ad4c1342219a1004181ac33ed8 at {username='slave'} at 127.3.139.72:53857: session id = f014c4ad4c1342219a1004181ac33ed8-f588a3ec49a24b1a9a9e19cdf8c30ca5
I20240502 07:59:54.431785 20955 tablet_copy_source_session.cc:205] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:54.432682 24767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f588a3ec49a24b1a9a9e19cdf8c30ca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:54.433072 20823 tablet_copy_service.cc:140] P c15e23a34059456788f9775031e27ec0: Received BeginTabletCopySession request for tablet 331368e52904494fa5ecf1ef594821fc from peer f014c4ad4c1342219a1004181ac33ed8 ({username='slave'} at 127.3.139.72:59525)
I20240502 07:59:54.433171 20823 tablet_copy_service.cc:161] P c15e23a34059456788f9775031e27ec0: Beginning new tablet copy session on tablet 331368e52904494fa5ecf1ef594821fc from peer f014c4ad4c1342219a1004181ac33ed8 at {username='slave'} at 127.3.139.72:59525: session id = f014c4ad4c1342219a1004181ac33ed8-331368e52904494fa5ecf1ef594821fc
I20240502 07:59:54.434017 20823 tablet_copy_source_session.cc:205] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:54.435281 24767 tablet_copy_client.cc:783] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:54.435428 24767 tablet_copy_client.cc:647] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:54.435950 24777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 331368e52904494fa5ecf1ef594821fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:54.437150 24767 tablet_copy_client.cc:523] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:54.437793 24777 tablet_copy_client.cc:783] T 331368e52904494fa5ecf1ef594821fc P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:54.437928 24777 tablet_copy_client.cc:647] T 331368e52904494fa5ecf1ef594821fc P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:54.438732 24767 tablet_bootstrap.cc:492] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P f014c4ad4c1342219a1004181ac33ed8: Bootstrap starting.
I20240502 07:59:54.438984 24777 tablet_copy_client.cc:523] T 331368e52904494fa5ecf1ef594821fc P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:54.440070 24777 tablet_bootstrap.cc:492] T 331368e52904494fa5ecf1ef594821fc P f014c4ad4c1342219a1004181ac33ed8: Bootstrap starting.
I20240502 07:59:54.442301 24790 ts_tablet_manager.cc:902] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P ff9f2ccccdfa48a1afbc0e371f41c662: Initiating tablet copy from peer e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315)
I20240502 07:59:54.445582 24798 ts_tablet_manager.cc:902] T e61fe3cb61c240f9b2c898aff1951144 P ff9f2ccccdfa48a1afbc0e371f41c662: Initiating tablet copy from peer 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587)
I20240502 07:59:54.446096 24798 tablet_copy_client.cc:308] T e61fe3cb61c240f9b2c898aff1951144 P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.67:45587
I20240502 07:59:54.450253 20427 tablet_copy_service.cc:140] P 35642aa2099c40e59bc2092604078e07: Received BeginTabletCopySession request for tablet e61fe3cb61c240f9b2c898aff1951144 from peer ff9f2ccccdfa48a1afbc0e371f41c662 ({username='slave'} at 127.3.139.74:34321)
I20240502 07:59:54.450361 20427 tablet_copy_service.cc:161] P 35642aa2099c40e59bc2092604078e07: Beginning new tablet copy session on tablet e61fe3cb61c240f9b2c898aff1951144 from peer ff9f2ccccdfa48a1afbc0e371f41c662 at {username='slave'} at 127.3.139.74:34321: session id = ff9f2ccccdfa48a1afbc0e371f41c662-e61fe3cb61c240f9b2c898aff1951144
I20240502 07:59:54.451264 20427 tablet_copy_source_session.cc:205] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:54.451687 24790 tablet_copy_client.cc:308] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.68:40315
I20240502 07:59:54.452150 24798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e61fe3cb61c240f9b2c898aff1951144. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:54.454205 24798 tablet_copy_client.cc:783] T e61fe3cb61c240f9b2c898aff1951144 P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:54.454336 24798 tablet_copy_client.cc:647] T e61fe3cb61c240f9b2c898aff1951144 P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:54.455663 24798 tablet_copy_client.cc:523] T e61fe3cb61c240f9b2c898aff1951144 P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:54.456804 24798 tablet_bootstrap.cc:492] T e61fe3cb61c240f9b2c898aff1951144 P ff9f2ccccdfa48a1afbc0e371f41c662: Bootstrap starting.
I20240502 07:59:54.458359 20559 tablet_copy_service.cc:140] P e951f009258e473cb57cf8773c7975d4: Received BeginTabletCopySession request for tablet dbf4b90cac9e4ff3b2936a1fa2ca8561 from peer ff9f2ccccdfa48a1afbc0e371f41c662 ({username='slave'} at 127.3.139.74:34453)
I20240502 07:59:54.458453 20559 tablet_copy_service.cc:161] P e951f009258e473cb57cf8773c7975d4: Beginning new tablet copy session on tablet dbf4b90cac9e4ff3b2936a1fa2ca8561 from peer ff9f2ccccdfa48a1afbc0e371f41c662 at {username='slave'} at 127.3.139.74:34453: session id = ff9f2ccccdfa48a1afbc0e371f41c662-dbf4b90cac9e4ff3b2936a1fa2ca8561
I20240502 07:59:54.459739 20559 tablet_copy_source_session.cc:205] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:54.460573 24790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbf4b90cac9e4ff3b2936a1fa2ca8561. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:54.462436 24790 tablet_copy_client.cc:783] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:54.462560 24790 tablet_copy_client.cc:647] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:54.463399 24767 tablet_bootstrap.cc:492] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P f014c4ad4c1342219a1004181ac33ed8: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:54.463724 24790 tablet_copy_client.cc:523] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:54.464670 24790 tablet_bootstrap.cc:492] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P ff9f2ccccdfa48a1afbc0e371f41c662: Bootstrap starting.
I20240502 07:59:54.467288 24767 tablet_bootstrap.cc:492] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P f014c4ad4c1342219a1004181ac33ed8: Bootstrap complete.
I20240502 07:59:54.467455 24767 ts_tablet_manager.cc:1367] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P f014c4ad4c1342219a1004181ac33ed8: Time spent bootstrapping tablet: real 0.029s user 0.022s sys 0.000s
I20240502 07:59:54.467882 24767 raft_consensus.cc:348] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } }
I20240502 07:59:54.468024 24767 raft_consensus.cc:729] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f014c4ad4c1342219a1004181ac33ed8, State: Initialized, Role: LEARNER
I20240502 07:59:54.468137 24767 consensus_queue.cc:260] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P f014c4ad4c1342219a1004181ac33ed8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } }
I20240502 07:59:54.468912 24767 ts_tablet_manager.cc:1398] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P f014c4ad4c1342219a1004181ac33ed8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:54.470217 20955 tablet_copy_service.cc:342] P b5f471f8b5944986a7d878cf6ac550b8: Request end of tablet copy session f014c4ad4c1342219a1004181ac33ed8-f588a3ec49a24b1a9a9e19cdf8c30ca5 received from {username='slave'} at 127.3.139.72:53857
I20240502 07:59:54.470285 20955 tablet_copy_service.cc:433] P b5f471f8b5944986a7d878cf6ac550b8: ending tablet copy session f014c4ad4c1342219a1004181ac33ed8-f588a3ec49a24b1a9a9e19cdf8c30ca5 on tablet f588a3ec49a24b1a9a9e19cdf8c30ca5 with peer f014c4ad4c1342219a1004181ac33ed8
I20240502 07:59:54.474133 24767 ts_tablet_manager.cc:902] T 1bc5937510214c3da7731d3ad2a67842 P f014c4ad4c1342219a1004181ac33ed8: Initiating tablet copy from peer b443e68b6212450286869124684bdfff (127.3.139.69:37923)
I20240502 07:59:54.474527 24767 tablet_copy_client.cc:308] T 1bc5937510214c3da7731d3ad2a67842 P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.69:37923
I20240502 07:59:54.475018 20691 tablet_copy_service.cc:140] P b443e68b6212450286869124684bdfff: Received BeginTabletCopySession request for tablet 1bc5937510214c3da7731d3ad2a67842 from peer f014c4ad4c1342219a1004181ac33ed8 ({username='slave'} at 127.3.139.72:49807)
I20240502 07:59:54.475153 20691 tablet_copy_service.cc:161] P b443e68b6212450286869124684bdfff: Beginning new tablet copy session on tablet 1bc5937510214c3da7731d3ad2a67842 from peer f014c4ad4c1342219a1004181ac33ed8 at {username='slave'} at 127.3.139.72:49807: session id = f014c4ad4c1342219a1004181ac33ed8-1bc5937510214c3da7731d3ad2a67842
I20240502 07:59:54.476032 20691 tablet_copy_source_session.cc:205] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:54.476898 24767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bc5937510214c3da7731d3ad2a67842. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:54.478900 24767 tablet_copy_client.cc:783] T 1bc5937510214c3da7731d3ad2a67842 P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:54.479024 24767 tablet_copy_client.cc:647] T 1bc5937510214c3da7731d3ad2a67842 P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:54.480195 24767 tablet_copy_client.cc:523] T 1bc5937510214c3da7731d3ad2a67842 P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:54.481288 24767 tablet_bootstrap.cc:492] T 1bc5937510214c3da7731d3ad2a67842 P f014c4ad4c1342219a1004181ac33ed8: Bootstrap starting.
I20240502 07:59:54.482110 24798 tablet_bootstrap.cc:492] T e61fe3cb61c240f9b2c898aff1951144 P ff9f2ccccdfa48a1afbc0e371f41c662: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:54.488718 24777 tablet_bootstrap.cc:492] T 331368e52904494fa5ecf1ef594821fc P f014c4ad4c1342219a1004181ac33ed8: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:54.488970 24777 tablet_bootstrap.cc:492] T 331368e52904494fa5ecf1ef594821fc P f014c4ad4c1342219a1004181ac33ed8: Bootstrap complete.
I20240502 07:59:54.489128 24777 ts_tablet_manager.cc:1367] T 331368e52904494fa5ecf1ef594821fc P f014c4ad4c1342219a1004181ac33ed8: Time spent bootstrapping tablet: real 0.049s user 0.016s sys 0.007s
I20240502 07:59:54.489526 24777 raft_consensus.cc:348] T 331368e52904494fa5ecf1ef594821fc P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } }
I20240502 07:59:54.489657 24777 raft_consensus.cc:729] T 331368e52904494fa5ecf1ef594821fc P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f014c4ad4c1342219a1004181ac33ed8, State: Initialized, Role: LEARNER
I20240502 07:59:54.489761 24777 consensus_queue.cc:260] T 331368e52904494fa5ecf1ef594821fc P f014c4ad4c1342219a1004181ac33ed8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } }
I20240502 07:59:54.490105 24777 ts_tablet_manager.cc:1398] T 331368e52904494fa5ecf1ef594821fc P f014c4ad4c1342219a1004181ac33ed8: Time spent starting tablet: real 0.001s user 0.002s sys 0.001s
I20240502 07:59:54.490859 20823 tablet_copy_service.cc:342] P c15e23a34059456788f9775031e27ec0: Request end of tablet copy session f014c4ad4c1342219a1004181ac33ed8-331368e52904494fa5ecf1ef594821fc received from {username='slave'} at 127.3.139.72:59525
I20240502 07:59:54.490932 20823 tablet_copy_service.cc:433] P c15e23a34059456788f9775031e27ec0: ending tablet copy session f014c4ad4c1342219a1004181ac33ed8-331368e52904494fa5ecf1ef594821fc on tablet 331368e52904494fa5ecf1ef594821fc with peer f014c4ad4c1342219a1004181ac33ed8
I20240502 07:59:54.495249 24798 tablet_bootstrap.cc:492] T e61fe3cb61c240f9b2c898aff1951144 P ff9f2ccccdfa48a1afbc0e371f41c662: Bootstrap complete.
I20240502 07:59:54.495430 24798 ts_tablet_manager.cc:1367] T e61fe3cb61c240f9b2c898aff1951144 P ff9f2ccccdfa48a1afbc0e371f41c662: Time spent bootstrapping tablet: real 0.039s user 0.025s sys 0.000s
I20240502 07:59:54.495888 24798 raft_consensus.cc:348] T e61fe3cb61c240f9b2c898aff1951144 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } }
I20240502 07:59:54.496028 24798 raft_consensus.cc:729] T e61fe3cb61c240f9b2c898aff1951144 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ff9f2ccccdfa48a1afbc0e371f41c662, State: Initialized, Role: LEARNER
I20240502 07:59:54.496125 24798 consensus_queue.cc:260] T e61fe3cb61c240f9b2c898aff1951144 P ff9f2ccccdfa48a1afbc0e371f41c662 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } }
I20240502 07:59:54.496487 24798 ts_tablet_manager.cc:1398] T e61fe3cb61c240f9b2c898aff1951144 P ff9f2ccccdfa48a1afbc0e371f41c662: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:54.496927 20427 tablet_copy_service.cc:342] P 35642aa2099c40e59bc2092604078e07: Request end of tablet copy session ff9f2ccccdfa48a1afbc0e371f41c662-e61fe3cb61c240f9b2c898aff1951144 received from {username='slave'} at 127.3.139.74:34321
I20240502 07:59:54.496999 20427 tablet_copy_service.cc:433] P 35642aa2099c40e59bc2092604078e07: ending tablet copy session ff9f2ccccdfa48a1afbc0e371f41c662-e61fe3cb61c240f9b2c898aff1951144 on tablet e61fe3cb61c240f9b2c898aff1951144 with peer ff9f2ccccdfa48a1afbc0e371f41c662
I20240502 07:59:54.503309 24767 tablet_bootstrap.cc:492] T 1bc5937510214c3da7731d3ad2a67842 P f014c4ad4c1342219a1004181ac33ed8: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:54.503551 24767 tablet_bootstrap.cc:492] T 1bc5937510214c3da7731d3ad2a67842 P f014c4ad4c1342219a1004181ac33ed8: Bootstrap complete.
I20240502 07:59:54.503718 24767 ts_tablet_manager.cc:1367] T 1bc5937510214c3da7731d3ad2a67842 P f014c4ad4c1342219a1004181ac33ed8: Time spent bootstrapping tablet: real 0.022s user 0.020s sys 0.002s
I20240502 07:59:54.504132 24767 raft_consensus.cc:348] T 1bc5937510214c3da7731d3ad2a67842 P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } }
I20240502 07:59:54.504253 24767 raft_consensus.cc:729] T 1bc5937510214c3da7731d3ad2a67842 P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f014c4ad4c1342219a1004181ac33ed8, State: Initialized, Role: LEARNER
I20240502 07:59:54.504354 24767 consensus_queue.cc:260] T 1bc5937510214c3da7731d3ad2a67842 P f014c4ad4c1342219a1004181ac33ed8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } }
I20240502 07:59:54.504640 24767 ts_tablet_manager.cc:1398] T 1bc5937510214c3da7731d3ad2a67842 P f014c4ad4c1342219a1004181ac33ed8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:54.505297 20691 tablet_copy_service.cc:342] P b443e68b6212450286869124684bdfff: Request end of tablet copy session f014c4ad4c1342219a1004181ac33ed8-1bc5937510214c3da7731d3ad2a67842 received from {username='slave'} at 127.3.139.72:49807
I20240502 07:59:54.505362 20691 tablet_copy_service.cc:433] P b443e68b6212450286869124684bdfff: ending tablet copy session f014c4ad4c1342219a1004181ac33ed8-1bc5937510214c3da7731d3ad2a67842 on tablet 1bc5937510214c3da7731d3ad2a67842 with peer f014c4ad4c1342219a1004181ac33ed8
I20240502 07:59:54.521504 24790 tablet_bootstrap.cc:492] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P ff9f2ccccdfa48a1afbc0e371f41c662: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:54.521816 24790 tablet_bootstrap.cc:492] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P ff9f2ccccdfa48a1afbc0e371f41c662: Bootstrap complete.
I20240502 07:59:54.522006 24790 ts_tablet_manager.cc:1367] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P ff9f2ccccdfa48a1afbc0e371f41c662: Time spent bootstrapping tablet: real 0.057s user 0.019s sys 0.000s
I20240502 07:59:54.522373 24790 raft_consensus.cc:348] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } }
I20240502 07:59:54.522487 24790 raft_consensus.cc:729] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ff9f2ccccdfa48a1afbc0e371f41c662, State: Initialized, Role: LEARNER
I20240502 07:59:54.522579 24790 consensus_queue.cc:260] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P ff9f2ccccdfa48a1afbc0e371f41c662 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } }
I20240502 07:59:54.522864 24790 ts_tablet_manager.cc:1398] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P ff9f2ccccdfa48a1afbc0e371f41c662: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240502 07:59:54.523422 20559 tablet_copy_service.cc:342] P e951f009258e473cb57cf8773c7975d4: Request end of tablet copy session ff9f2ccccdfa48a1afbc0e371f41c662-dbf4b90cac9e4ff3b2936a1fa2ca8561 received from {username='slave'} at 127.3.139.74:34453
I20240502 07:59:54.523509 20559 tablet_copy_service.cc:433] P e951f009258e473cb57cf8773c7975d4: ending tablet copy session ff9f2ccccdfa48a1afbc0e371f41c662-dbf4b90cac9e4ff3b2936a1fa2ca8561 on tablet dbf4b90cac9e4ff3b2936a1fa2ca8561 with peer ff9f2ccccdfa48a1afbc0e371f41c662
I20240502 07:59:54.560127 24790 ts_tablet_manager.cc:902] T 1453e833d0ad494583c44b5496088f6d P ff9f2ccccdfa48a1afbc0e371f41c662: Initiating tablet copy from peer b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577)
I20240502 07:59:54.560561 24790 tablet_copy_client.cc:308] T 1453e833d0ad494583c44b5496088f6d P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.71:39577
I20240502 07:59:54.560972 20955 tablet_copy_service.cc:140] P b5f471f8b5944986a7d878cf6ac550b8: Received BeginTabletCopySession request for tablet 1453e833d0ad494583c44b5496088f6d from peer ff9f2ccccdfa48a1afbc0e371f41c662 ({username='slave'} at 127.3.139.74:54441)
I20240502 07:59:54.561089 20955 tablet_copy_service.cc:161] P b5f471f8b5944986a7d878cf6ac550b8: Beginning new tablet copy session on tablet 1453e833d0ad494583c44b5496088f6d from peer ff9f2ccccdfa48a1afbc0e371f41c662 at {username='slave'} at 127.3.139.74:54441: session id = ff9f2ccccdfa48a1afbc0e371f41c662-1453e833d0ad494583c44b5496088f6d
I20240502 07:59:54.561827 20955 tablet_copy_source_session.cc:205] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:54.562724 24790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1453e833d0ad494583c44b5496088f6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:54.564347 24790 tablet_copy_client.cc:783] T 1453e833d0ad494583c44b5496088f6d P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:54.564474 24790 tablet_copy_client.cc:647] T 1453e833d0ad494583c44b5496088f6d P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:54.565565 24790 tablet_copy_client.cc:523] T 1453e833d0ad494583c44b5496088f6d P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:54.566531 24790 tablet_bootstrap.cc:492] T 1453e833d0ad494583c44b5496088f6d P ff9f2ccccdfa48a1afbc0e371f41c662: Bootstrap starting.
I20240502 07:59:54.568300 24798 ts_tablet_manager.cc:902] T 9ad40762dc784dfa9f3507e025db32b8 P ff9f2ccccdfa48a1afbc0e371f41c662: Initiating tablet copy from peer 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703)
I20240502 07:59:54.568800 24798 tablet_copy_client.cc:308] T 9ad40762dc784dfa9f3507e025db32b8 P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.65:44703
I20240502 07:59:54.569257 20163 tablet_copy_service.cc:140] P 29a7180fe4574c09a8dce7c27500aab2: Received BeginTabletCopySession request for tablet 9ad40762dc784dfa9f3507e025db32b8 from peer ff9f2ccccdfa48a1afbc0e371f41c662 ({username='slave'} at 127.3.139.74:47167)
I20240502 07:59:54.569357 20163 tablet_copy_service.cc:161] P 29a7180fe4574c09a8dce7c27500aab2: Beginning new tablet copy session on tablet 9ad40762dc784dfa9f3507e025db32b8 from peer ff9f2ccccdfa48a1afbc0e371f41c662 at {username='slave'} at 127.3.139.74:47167: session id = ff9f2ccccdfa48a1afbc0e371f41c662-9ad40762dc784dfa9f3507e025db32b8
I20240502 07:59:54.570233 20163 tablet_copy_source_session.cc:205] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:54.570989 24798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ad40762dc784dfa9f3507e025db32b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:54.572763 24798 tablet_copy_client.cc:783] T 9ad40762dc784dfa9f3507e025db32b8 P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:54.572988 24798 tablet_copy_client.cc:647] T 9ad40762dc784dfa9f3507e025db32b8 P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:54.574127 24798 tablet_copy_client.cc:523] T 9ad40762dc784dfa9f3507e025db32b8 P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:54.575213 24798 tablet_bootstrap.cc:492] T 9ad40762dc784dfa9f3507e025db32b8 P ff9f2ccccdfa48a1afbc0e371f41c662: Bootstrap starting.
I20240502 07:59:54.577659 24767 ts_tablet_manager.cc:902] T 15bb9480e88b41219cc91168cc438c70 P f014c4ad4c1342219a1004181ac33ed8: Initiating tablet copy from peer b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577)
I20240502 07:59:54.578100 24767 tablet_copy_client.cc:308] T 15bb9480e88b41219cc91168cc438c70 P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.71:39577
I20240502 07:59:54.578541 20955 tablet_copy_service.cc:140] P b5f471f8b5944986a7d878cf6ac550b8: Received BeginTabletCopySession request for tablet 15bb9480e88b41219cc91168cc438c70 from peer f014c4ad4c1342219a1004181ac33ed8 ({username='slave'} at 127.3.139.72:53857)
I20240502 07:59:54.578634 20955 tablet_copy_service.cc:161] P b5f471f8b5944986a7d878cf6ac550b8: Beginning new tablet copy session on tablet 15bb9480e88b41219cc91168cc438c70 from peer f014c4ad4c1342219a1004181ac33ed8 at {username='slave'} at 127.3.139.72:53857: session id = f014c4ad4c1342219a1004181ac33ed8-15bb9480e88b41219cc91168cc438c70
I20240502 07:59:54.579342 20955 tablet_copy_source_session.cc:205] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:54.580545 24767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15bb9480e88b41219cc91168cc438c70. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:54.582582 24767 tablet_copy_client.cc:783] T 15bb9480e88b41219cc91168cc438c70 P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:54.582706 24767 tablet_copy_client.cc:647] T 15bb9480e88b41219cc91168cc438c70 P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:54.592077 24767 tablet_copy_client.cc:523] T 15bb9480e88b41219cc91168cc438c70 P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:54.593133 24767 tablet_bootstrap.cc:492] T 15bb9480e88b41219cc91168cc438c70 P f014c4ad4c1342219a1004181ac33ed8: Bootstrap starting.
I20240502 07:59:54.593202 24790 tablet_bootstrap.cc:492] T 1453e833d0ad494583c44b5496088f6d P ff9f2ccccdfa48a1afbc0e371f41c662: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:54.593443 24790 tablet_bootstrap.cc:492] T 1453e833d0ad494583c44b5496088f6d P ff9f2ccccdfa48a1afbc0e371f41c662: Bootstrap complete.
I20240502 07:59:54.593609 24790 ts_tablet_manager.cc:1367] T 1453e833d0ad494583c44b5496088f6d P ff9f2ccccdfa48a1afbc0e371f41c662: Time spent bootstrapping tablet: real 0.027s user 0.025s sys 0.000s
I20240502 07:59:54.594023 24790 raft_consensus.cc:348] T 1453e833d0ad494583c44b5496088f6d P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } }
I20240502 07:59:54.594161 24790 raft_consensus.cc:729] T 1453e833d0ad494583c44b5496088f6d P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ff9f2ccccdfa48a1afbc0e371f41c662, State: Initialized, Role: LEARNER
I20240502 07:59:54.594260 24790 consensus_queue.cc:260] T 1453e833d0ad494583c44b5496088f6d P ff9f2ccccdfa48a1afbc0e371f41c662 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } }
I20240502 07:59:54.595268 24790 ts_tablet_manager.cc:1398] T 1453e833d0ad494583c44b5496088f6d P ff9f2ccccdfa48a1afbc0e371f41c662: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20240502 07:59:54.595710 20955 tablet_copy_service.cc:342] P b5f471f8b5944986a7d878cf6ac550b8: Request end of tablet copy session ff9f2ccccdfa48a1afbc0e371f41c662-1453e833d0ad494583c44b5496088f6d received from {username='slave'} at 127.3.139.74:54441
I20240502 07:59:54.595779 20955 tablet_copy_service.cc:433] P b5f471f8b5944986a7d878cf6ac550b8: ending tablet copy session ff9f2ccccdfa48a1afbc0e371f41c662-1453e833d0ad494583c44b5496088f6d on tablet 1453e833d0ad494583c44b5496088f6d with peer ff9f2ccccdfa48a1afbc0e371f41c662
I20240502 07:59:54.604684 24798 tablet_bootstrap.cc:492] T 9ad40762dc784dfa9f3507e025db32b8 P ff9f2ccccdfa48a1afbc0e371f41c662: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:54.604907 24798 tablet_bootstrap.cc:492] T 9ad40762dc784dfa9f3507e025db32b8 P ff9f2ccccdfa48a1afbc0e371f41c662: Bootstrap complete.
I20240502 07:59:54.605057 24798 ts_tablet_manager.cc:1367] T 9ad40762dc784dfa9f3507e025db32b8 P ff9f2ccccdfa48a1afbc0e371f41c662: Time spent bootstrapping tablet: real 0.030s user 0.022s sys 0.000s
I20240502 07:59:54.605384 24798 raft_consensus.cc:348] T 9ad40762dc784dfa9f3507e025db32b8 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } }
I20240502 07:59:54.605489 24798 raft_consensus.cc:729] T 9ad40762dc784dfa9f3507e025db32b8 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ff9f2ccccdfa48a1afbc0e371f41c662, State: Initialized, Role: LEARNER
I20240502 07:59:54.605577 24798 consensus_queue.cc:260] T 9ad40762dc784dfa9f3507e025db32b8 P ff9f2ccccdfa48a1afbc0e371f41c662 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } }
I20240502 07:59:54.605908 24798 ts_tablet_manager.cc:1398] T 9ad40762dc784dfa9f3507e025db32b8 P ff9f2ccccdfa48a1afbc0e371f41c662: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240502 07:59:54.606563 20163 tablet_copy_service.cc:342] P 29a7180fe4574c09a8dce7c27500aab2: Request end of tablet copy session ff9f2ccccdfa48a1afbc0e371f41c662-9ad40762dc784dfa9f3507e025db32b8 received from {username='slave'} at 127.3.139.74:47167
I20240502 07:59:54.606637 20163 tablet_copy_service.cc:433] P 29a7180fe4574c09a8dce7c27500aab2: ending tablet copy session ff9f2ccccdfa48a1afbc0e371f41c662-9ad40762dc784dfa9f3507e025db32b8 on tablet 9ad40762dc784dfa9f3507e025db32b8 with peer ff9f2ccccdfa48a1afbc0e371f41c662
I20240502 07:59:54.613595 24777 ts_tablet_manager.cc:902] T 0e2908de10684c84af542f16ff8727cb P f014c4ad4c1342219a1004181ac33ed8: Initiating tablet copy from peer b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331)
I20240502 07:59:54.614061 24777 tablet_copy_client.cc:308] T 0e2908de10684c84af542f16ff8727cb P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.66:45331
I20240502 07:59:54.614480 20295 tablet_copy_service.cc:140] P b7604c0eb3bf4a7cb260b564a94a1191: Received BeginTabletCopySession request for tablet 0e2908de10684c84af542f16ff8727cb from peer f014c4ad4c1342219a1004181ac33ed8 ({username='slave'} at 127.3.139.72:46325)
I20240502 07:59:54.614590 20295 tablet_copy_service.cc:161] P b7604c0eb3bf4a7cb260b564a94a1191: Beginning new tablet copy session on tablet 0e2908de10684c84af542f16ff8727cb from peer f014c4ad4c1342219a1004181ac33ed8 at {username='slave'} at 127.3.139.72:46325: session id = f014c4ad4c1342219a1004181ac33ed8-0e2908de10684c84af542f16ff8727cb
I20240502 07:59:54.615538 20295 tablet_copy_source_session.cc:205] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:54.616369 24777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e2908de10684c84af542f16ff8727cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:54.617980 24777 tablet_copy_client.cc:783] T 0e2908de10684c84af542f16ff8727cb P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:54.618104 24777 tablet_copy_client.cc:647] T 0e2908de10684c84af542f16ff8727cb P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:54.618359 24767 tablet_bootstrap.cc:492] T 15bb9480e88b41219cc91168cc438c70 P f014c4ad4c1342219a1004181ac33ed8: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:54.618628 24767 tablet_bootstrap.cc:492] T 15bb9480e88b41219cc91168cc438c70 P f014c4ad4c1342219a1004181ac33ed8: Bootstrap complete.
I20240502 07:59:54.618781 24767 ts_tablet_manager.cc:1367] T 15bb9480e88b41219cc91168cc438c70 P f014c4ad4c1342219a1004181ac33ed8: Time spent bootstrapping tablet: real 0.026s user 0.023s sys 0.000s
I20240502 07:59:54.619179 24777 tablet_copy_client.cc:523] T 0e2908de10684c84af542f16ff8727cb P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:54.619199 24767 raft_consensus.cc:348] T 15bb9480e88b41219cc91168cc438c70 P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } }
I20240502 07:59:54.619340 24767 raft_consensus.cc:729] T 15bb9480e88b41219cc91168cc438c70 P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f014c4ad4c1342219a1004181ac33ed8, State: Initialized, Role: LEARNER
I20240502 07:59:54.619446 24767 consensus_queue.cc:260] T 15bb9480e88b41219cc91168cc438c70 P f014c4ad4c1342219a1004181ac33ed8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } }
I20240502 07:59:54.620162 24767 ts_tablet_manager.cc:1398] T 15bb9480e88b41219cc91168cc438c70 P f014c4ad4c1342219a1004181ac33ed8: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240502 07:59:54.620298 24777 tablet_bootstrap.cc:492] T 0e2908de10684c84af542f16ff8727cb P f014c4ad4c1342219a1004181ac33ed8: Bootstrap starting.
I20240502 07:59:54.620604 20955 tablet_copy_service.cc:342] P b5f471f8b5944986a7d878cf6ac550b8: Request end of tablet copy session f014c4ad4c1342219a1004181ac33ed8-15bb9480e88b41219cc91168cc438c70 received from {username='slave'} at 127.3.139.72:53857
I20240502 07:59:54.620693 20955 tablet_copy_service.cc:433] P b5f471f8b5944986a7d878cf6ac550b8: ending tablet copy session f014c4ad4c1342219a1004181ac33ed8-15bb9480e88b41219cc91168cc438c70 on tablet 15bb9480e88b41219cc91168cc438c70 with peer f014c4ad4c1342219a1004181ac33ed8
I20240502 07:59:54.638315 24777 tablet_bootstrap.cc:492] T 0e2908de10684c84af542f16ff8727cb P f014c4ad4c1342219a1004181ac33ed8: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:54.638543 24777 tablet_bootstrap.cc:492] T 0e2908de10684c84af542f16ff8727cb P f014c4ad4c1342219a1004181ac33ed8: Bootstrap complete.
I20240502 07:59:54.638697 24777 ts_tablet_manager.cc:1367] T 0e2908de10684c84af542f16ff8727cb P f014c4ad4c1342219a1004181ac33ed8: Time spent bootstrapping tablet: real 0.018s user 0.014s sys 0.001s
I20240502 07:59:54.639117 24777 raft_consensus.cc:348] T 0e2908de10684c84af542f16ff8727cb P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } }
I20240502 07:59:54.639235 24777 raft_consensus.cc:729] T 0e2908de10684c84af542f16ff8727cb P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f014c4ad4c1342219a1004181ac33ed8, State: Initialized, Role: LEARNER
I20240502 07:59:54.639320 24777 consensus_queue.cc:260] T 0e2908de10684c84af542f16ff8727cb P f014c4ad4c1342219a1004181ac33ed8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } }
I20240502 07:59:54.639612 24777 ts_tablet_manager.cc:1398] T 0e2908de10684c84af542f16ff8727cb P f014c4ad4c1342219a1004181ac33ed8: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240502 07:59:54.640105 20295 tablet_copy_service.cc:342] P b7604c0eb3bf4a7cb260b564a94a1191: Request end of tablet copy session f014c4ad4c1342219a1004181ac33ed8-0e2908de10684c84af542f16ff8727cb received from {username='slave'} at 127.3.139.72:46325
I20240502 07:59:54.640187 20295 tablet_copy_service.cc:433] P b7604c0eb3bf4a7cb260b564a94a1191: ending tablet copy session f014c4ad4c1342219a1004181ac33ed8-0e2908de10684c84af542f16ff8727cb on tablet 0e2908de10684c84af542f16ff8727cb with peer f014c4ad4c1342219a1004181ac33ed8
I20240502 07:59:54.644276 24798 ts_tablet_manager.cc:902] T ae6d597fbc9242539edee8685a015d99 P ff9f2ccccdfa48a1afbc0e371f41c662: Initiating tablet copy from peer b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331)
I20240502 07:59:54.644696 24798 tablet_copy_client.cc:308] T ae6d597fbc9242539edee8685a015d99 P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.66:45331
I20240502 07:59:54.647966 20295 tablet_copy_service.cc:140] P b7604c0eb3bf4a7cb260b564a94a1191: Received BeginTabletCopySession request for tablet ae6d597fbc9242539edee8685a015d99 from peer ff9f2ccccdfa48a1afbc0e371f41c662 ({username='slave'} at 127.3.139.74:51475)
I20240502 07:59:54.648046 20295 tablet_copy_service.cc:161] P b7604c0eb3bf4a7cb260b564a94a1191: Beginning new tablet copy session on tablet ae6d597fbc9242539edee8685a015d99 from peer ff9f2ccccdfa48a1afbc0e371f41c662 at {username='slave'} at 127.3.139.74:51475: session id = ff9f2ccccdfa48a1afbc0e371f41c662-ae6d597fbc9242539edee8685a015d99
I20240502 07:59:54.648711 20295 tablet_copy_source_session.cc:205] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:54.649420 24798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae6d597fbc9242539edee8685a015d99. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:54.650967 24798 tablet_copy_client.cc:783] T ae6d597fbc9242539edee8685a015d99 P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:54.651125 24798 tablet_copy_client.cc:647] T ae6d597fbc9242539edee8685a015d99 P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:54.652060 24798 tablet_copy_client.cc:523] T ae6d597fbc9242539edee8685a015d99 P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:54.652951 24798 tablet_bootstrap.cc:492] T ae6d597fbc9242539edee8685a015d99 P ff9f2ccccdfa48a1afbc0e371f41c662: Bootstrap starting.
I20240502 07:59:54.677090 24798 tablet_bootstrap.cc:492] T ae6d597fbc9242539edee8685a015d99 P ff9f2ccccdfa48a1afbc0e371f41c662: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:54.677353 24798 tablet_bootstrap.cc:492] T ae6d597fbc9242539edee8685a015d99 P ff9f2ccccdfa48a1afbc0e371f41c662: Bootstrap complete.
I20240502 07:59:54.677548 24798 ts_tablet_manager.cc:1367] T ae6d597fbc9242539edee8685a015d99 P ff9f2ccccdfa48a1afbc0e371f41c662: Time spent bootstrapping tablet: real 0.025s user 0.015s sys 0.006s
I20240502 07:59:54.677994 24798 raft_consensus.cc:348] T ae6d597fbc9242539edee8685a015d99 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } }
I20240502 07:59:54.678145 24798 raft_consensus.cc:729] T ae6d597fbc9242539edee8685a015d99 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ff9f2ccccdfa48a1afbc0e371f41c662, State: Initialized, Role: LEARNER
I20240502 07:59:54.678254 24798 consensus_queue.cc:260] T ae6d597fbc9242539edee8685a015d99 P ff9f2ccccdfa48a1afbc0e371f41c662 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } }
I20240502 07:59:54.678735 24798 ts_tablet_manager.cc:1398] T ae6d597fbc9242539edee8685a015d99 P ff9f2ccccdfa48a1afbc0e371f41c662: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240502 07:59:54.679356 20295 tablet_copy_service.cc:342] P b7604c0eb3bf4a7cb260b564a94a1191: Request end of tablet copy session ff9f2ccccdfa48a1afbc0e371f41c662-ae6d597fbc9242539edee8685a015d99 received from {username='slave'} at 127.3.139.74:51475
I20240502 07:59:54.679436 20295 tablet_copy_service.cc:433] P b7604c0eb3bf4a7cb260b564a94a1191: ending tablet copy session ff9f2ccccdfa48a1afbc0e371f41c662-ae6d597fbc9242539edee8685a015d99 on tablet ae6d597fbc9242539edee8685a015d99 with peer ff9f2ccccdfa48a1afbc0e371f41c662
I20240502 07:59:54.911731 22609 raft_consensus.cc:1203] T e61fe3cb61c240f9b2c898aff1951144 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240502 07:59:54.915791 22741 raft_consensus.cc:1203] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240502 07:59:54.916214 22609 raft_consensus.cc:1203] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240502 07:59:54.960379 22741 raft_consensus.cc:1203] T 331368e52904494fa5ecf1ef594821fc P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240502 07:59:55.066170 22741 raft_consensus.cc:1203] T 1bc5937510214c3da7731d3ad2a67842 P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240502 07:59:55.082895 22609 raft_consensus.cc:1203] T 1453e833d0ad494583c44b5496088f6d P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240502 07:59:55.100833 22609 raft_consensus.cc:1203] T 9ad40762dc784dfa9f3507e025db32b8 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240502 07:59:55.158505 22741 raft_consensus.cc:1203] T 15bb9480e88b41219cc91168cc438c70 P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240502 07:59:55.203506 22741 raft_consensus.cc:1203] T 0e2908de10684c84af542f16ff8727cb P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240502 07:59:55.211184 22609 raft_consensus.cc:1203] T ae6d597fbc9242539edee8685a015d99 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240502 07:59:55.312178 24901 raft_consensus.cc:1050] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07: attempting to promote NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 to VOTER
I20240502 07:59:55.312693 24901 consensus_queue.cc:237] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } }
I20240502 07:59:55.314654 20801 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.314656 20539 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.314805 20935 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.315034 22475 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.315127 25011 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.315321 25010 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.315570 25012 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.315903 20272 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.315908 22609 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.316511 24900 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.316664 25011 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.316882 25009 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.317209 20142 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.317660 25009 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.319156 24900 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.320037 22475 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.320499 20142 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.321635 20801 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.321755 20539 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.322898 20935 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.323239 19997 catalog_manager.cc:5484] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 reported cstate change: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "35642aa2099c40e59bc2092604078e07" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:55.324090 22609 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.328615 20272 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.331658 20406 consensus_queue.cc:237] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } }
I20240502 07:59:55.333340 20539 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.333366 22609 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.333504 20801 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.334029 25010 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.334175 24900 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.334292 25009 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.335971 20935 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.336099 22475 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.336339 25012 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.336470 20272 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.336829 25012 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.337218 24900 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.337823 25010 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.338153 22609 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.338814 20935 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.339040 22475 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.340013 20801 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.340112 20539 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.342797 19988 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet e61fe3cb61c240f9b2c898aff1951144 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:55.343109 20272 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.343534 19999 catalog_manager.cc:5484] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 reported cstate change: config changed from index 9 to 10, VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) evicted. New cstate: current_term: 1 leader_uuid: "35642aa2099c40e59bc2092604078e07" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:55.351329 20122 tablet_service.cc:1514] Processing DeleteTablet for tablet e61fe3cb61c240f9b2c898aff1951144 with delete_type TABLET_DATA_TOMBSTONED (TS 29a7180fe4574c09a8dce7c27500aab2 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:44498
I20240502 07:59:55.351814 25025 tablet_replica.cc:330] T e61fe3cb61c240f9b2c898aff1951144 P 29a7180fe4574c09a8dce7c27500aab2: stopping tablet replica
I20240502 07:59:55.352185 25025 raft_consensus.cc:2229] T e61fe3cb61c240f9b2c898aff1951144 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:55.352488 25025 raft_consensus.cc:2258] T e61fe3cb61c240f9b2c898aff1951144 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:55.354107 25025 ts_tablet_manager.cc:1875] T e61fe3cb61c240f9b2c898aff1951144 P 29a7180fe4574c09a8dce7c27500aab2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:55.356097 25025 ts_tablet_manager.cc:1888] T e61fe3cb61c240f9b2c898aff1951144 P 29a7180fe4574c09a8dce7c27500aab2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240502 07:59:55.356168 25025 log.cc:1196] T e61fe3cb61c240f9b2c898aff1951144 P 29a7180fe4574c09a8dce7c27500aab2: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal/wals/e61fe3cb61c240f9b2c898aff1951144
I20240502 07:59:55.356606 19986 catalog_manager.cc:4830] TS 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703): tablet e61fe3cb61c240f9b2c898aff1951144 (table rebalance_test_table_3 [id=1388635dd26f4d3791ee92dfbf8d374a]) successfully deleted
I20240502 07:59:55.402407 24943 raft_consensus.cc:1050] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0: attempting to promote NON_VOTER f014c4ad4c1342219a1004181ac33ed8 to VOTER
I20240502 07:59:55.402889 24943 consensus_queue.cc:237] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } }
I20240502 07:59:55.404716 22741 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.404848 20935 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.404848 20539 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.404865 22475 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.405261 25029 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.405342 20142 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.405436 20272 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.405462 25028 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.405704 25027 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.405849 24533 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.406260 24943 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.406445 25029 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.406870 20406 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.407675 25029 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.409401 25027 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.409842 22741 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.410248 20406 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.411917 20272 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.412385 22475 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.413651 20000 catalog_manager.cc:5484] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 reported cstate change: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c15e23a34059456788f9775031e27ec0" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:55.414539 20142 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.415028 20539 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.423985 20935 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.427296 20800 consensus_queue.cc:237] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } }
I20240502 07:59:55.429435 22741 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.429809 25028 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.430958 22475 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.431277 20272 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.431339 25030 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.431859 24533 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.432673 20406 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.433056 20935 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.433492 20539 raft_consensus.cc:1261] T 331368e52904494fa5ecf1ef594821fc P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.433625 25030 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.433897 24533 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.434896 25028 consensus_queue.cc:1035] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.435581 24533 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.436923 20406 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.437067 20935 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.438011 19997 catalog_manager.cc:5484] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 reported cstate change: config changed from index 9 to 10, VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) evicted. New cstate: current_term: 1 leader_uuid: "c15e23a34059456788f9775031e27ec0" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:55.439415 19986 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 331368e52904494fa5ecf1ef594821fc with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:55.439888 22741 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.440193 20539 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.441668 20122 tablet_service.cc:1514] Processing DeleteTablet for tablet 331368e52904494fa5ecf1ef594821fc with delete_type TABLET_DATA_TOMBSTONED (TS 29a7180fe4574c09a8dce7c27500aab2 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:44498
I20240502 07:59:55.441870 25025 tablet_replica.cc:330] T 331368e52904494fa5ecf1ef594821fc P 29a7180fe4574c09a8dce7c27500aab2: stopping tablet replica
I20240502 07:59:55.441715 20275 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.441982 25025 raft_consensus.cc:2229] T 331368e52904494fa5ecf1ef594821fc P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:55.442200 25025 raft_consensus.cc:2258] T 331368e52904494fa5ecf1ef594821fc P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:55.442709 22475 raft_consensus.cc:2932] T 331368e52904494fa5ecf1ef594821fc P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.447589 25025 ts_tablet_manager.cc:1875] T 331368e52904494fa5ecf1ef594821fc P 29a7180fe4574c09a8dce7c27500aab2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:55.449681 25025 ts_tablet_manager.cc:1888] T 331368e52904494fa5ecf1ef594821fc P 29a7180fe4574c09a8dce7c27500aab2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240502 07:59:55.449774 25025 log.cc:1196] T 331368e52904494fa5ecf1ef594821fc P 29a7180fe4574c09a8dce7c27500aab2: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal/wals/331368e52904494fa5ecf1ef594821fc
I20240502 07:59:55.450248 19986 catalog_manager.cc:4830] TS 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703): tablet 331368e52904494fa5ecf1ef594821fc (table rebalance_test_table_4 [id=8edc4db66dcd46ee8429441e0ce671a3]) successfully deleted
I20240502 07:59:55.455328 24601 raft_consensus.cc:1050] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8: attempting to promote NON_VOTER f014c4ad4c1342219a1004181ac33ed8 to VOTER
I20240502 07:59:55.455888 24601 consensus_queue.cc:237] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } }
I20240502 07:59:55.457795 20800 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.457811 22741 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.457964 20539 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.458103 20406 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.458470 20142 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.459431 22475 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.459760 25054 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.459944 20275 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.460062 24601 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.460211 24902 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.460564 24902 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.460687 24618 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.460981 24601 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.461627 25055 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.465690 25055 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.466193 20272 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.466912 24902 raft_consensus.cc:1050] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8: attempting to promote NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 to VOTER
I20240502 07:59:55.467435 24902 consensus_queue.cc:237] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } }
I20240502 07:59:55.467481 20406 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.468346 22475 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.468930 19999 catalog_manager.cc:5484] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 reported cstate change: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b5f471f8b5944986a7d878cf6ac550b8" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:55.470312 20539 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.470829 22475 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.470887 20668 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.470961 22741 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.471192 24618 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.471382 24601 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.471778 20142 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.473145 20405 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.473526 25054 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.474363 20272 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.474586 22609 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.475456 25057 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.475536 20539 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.476267 25074 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.477692 20800 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.479818 25054 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.480068 25057 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.480823 20142 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.482395 20935 consensus_queue.cc:237] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } }
I20240502 07:59:55.484127 22475 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.484511 20800 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.484544 22741 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.484612 20537 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.484967 24618 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.485056 20275 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.485093 25055 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.485189 24601 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.485451 24601 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.486019 24556 raft_consensus.cc:1050] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4: attempting to promote NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 to VOTER
I20240502 07:59:55.486506 24556 consensus_queue.cc:237] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } }
I20240502 07:59:55.488027 25070 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.488137 22475 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.488320 25074 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.489056 20142 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.489506 24618 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.490168 25055 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.490164 20405 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.491196 22609 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.492133 20537 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.492512 22476 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.492941 19988 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet f588a3ec49a24b1a9a9e19cdf8c30ca5 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:55.493520 20800 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.493567 20933 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.494000 20143 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.494042 20800 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.494630 19992 catalog_manager.cc:5484] T 1453e833d0ad494583c44b5496088f6d P 35642aa2099c40e59bc2092604078e07 reported cstate change: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b5f471f8b5944986a7d878cf6ac550b8" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.494922 24557 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.495313 24557 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.496600 22741 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.496786 20668 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.497007 20273 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.496879 20142 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.497721 20143 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.498406 25091 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.499277 19999 catalog_manager.cc:5484] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P f014c4ad4c1342219a1004181ac33ed8 reported cstate change: config changed from index 9 to 10, VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) evicted. New cstate: current_term: 1 leader_uuid: "b5f471f8b5944986a7d878cf6ac550b8" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.499689 25091 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.500147 24556 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.501313 22593 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.502605 20386 tablet_service.cc:1514] Processing DeleteTablet for tablet f588a3ec49a24b1a9a9e19cdf8c30ca5 with delete_type TABLET_DATA_TOMBSTONED (TS 35642aa2099c40e59bc2092604078e07 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:43238
I20240502 07:59:55.503147 25097 tablet_replica.cc:330] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07: stopping tablet replica
I20240502 07:59:55.503479 25097 raft_consensus.cc:2229] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:55.503719 25097 raft_consensus.cc:2258] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:55.505086 25097 ts_tablet_manager.cc:1875] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:55.506105 20537 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.506605 20405 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.506989 24557 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.507198 20933 consensus_queue.cc:237] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } }
I20240502 07:59:55.509140 20406 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.509546 25054 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.510540 20668 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.511138 22476 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.511039 20273 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.511267 20143 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.511531 25054 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.511711 25055 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.513015 22593 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.513458 24902 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.513782 24902 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.514870 25097 ts_tablet_manager.cc:1888] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240502 07:59:55.515115 25097 log.cc:1196] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal/wals/f588a3ec49a24b1a9a9e19cdf8c30ca5
I20240502 07:59:55.515516 22475 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.516008 19988 catalog_manager.cc:4830] TS 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587): tablet f588a3ec49a24b1a9a9e19cdf8c30ca5 (table rebalance_test_table_2 [id=30a30b02a2404507a95fc3b5bb77bdd1]) successfully deleted
I20240502 07:59:55.517906 20537 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.517969 24557 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.518944 20668 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.519264 24618 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.520968 19988 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 1453e833d0ad494583c44b5496088f6d with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:55.521378 24902 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.521344 22476 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.521476 20405 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.522763 20537 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.523243 20143 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.523962 19993 catalog_manager.cc:5484] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 reported cstate change: config changed from index 9 to 10, VOTER b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) evicted. New cstate: current_term: 1 leader_uuid: "b5f471f8b5944986a7d878cf6ac550b8" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:55.526347 22593 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.526705 25093 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.527727 20253 tablet_service.cc:1514] Processing DeleteTablet for tablet 1453e833d0ad494583c44b5496088f6d with delete_type TABLET_DATA_TOMBSTONED (TS b7604c0eb3bf4a7cb260b564a94a1191 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:59920
I20240502 07:59:55.529132 22475 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.530176 20273 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.532589 19994 catalog_manager.cc:5484] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b7604c0eb3bf4a7cb260b564a94a1191 reported cstate change: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e951f009258e473cb57cf8773c7975d4" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.533061 22593 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.533195 20143 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.533504 20405 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.535897 20935 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.537113 25102 tablet_replica.cc:330] T 1453e833d0ad494583c44b5496088f6d P b7604c0eb3bf4a7cb260b564a94a1191: stopping tablet replica
I20240502 07:59:55.537470 25102 raft_consensus.cc:2229] T 1453e833d0ad494583c44b5496088f6d P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:55.537765 25102 pending_rounds.cc:62] T 1453e833d0ad494583c44b5496088f6d P b7604c0eb3bf4a7cb260b564a94a1191: Trying to abort 1 pending ops.
I20240502 07:59:55.537843 25102 pending_rounds.cc:68] T 1453e833d0ad494583c44b5496088f6d P b7604c0eb3bf4a7cb260b564a94a1191: Aborting op as it isn't in flight: id { term: 1 index: 9 } timestamp: 7023152314233159680 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "1453e833d0ad494583c44b5496088f6d" old_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } } new_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } } }
I20240502 07:59:55.538048 25102 raft_consensus.cc:2894] T 1453e833d0ad494583c44b5496088f6d P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Aborting config change with OpId 1.9: Aborted: Op aborted
I20240502 07:59:55.538120 25102 raft_consensus.cc:2866] T 1453e833d0ad494583c44b5496088f6d P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20240502 07:59:55.538198 25102 raft_consensus.cc:2258] T 1453e833d0ad494583c44b5496088f6d P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Raft consensus is shut down!
W20240502 07:59:55.539750 25081 consensus_queue.cc:1159] T 1453e833d0ad494583c44b5496088f6d P b7604c0eb3bf4a7cb260b564a94a1191 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" status { last_received { term: 1 index: 9 } last_committed_idx: 8 last_received_current_leader { term: 1 index: 9 } }
I20240502 07:59:55.541245 25102 ts_tablet_manager.cc:1875] T 1453e833d0ad494583c44b5496088f6d P b7604c0eb3bf4a7cb260b564a94a1191: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:55.541424 20800 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.543624 25102 ts_tablet_manager.cc:1888] T 1453e833d0ad494583c44b5496088f6d P b7604c0eb3bf4a7cb260b564a94a1191: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240502 07:59:55.543752 25102 log.cc:1196] T 1453e833d0ad494583c44b5496088f6d P b7604c0eb3bf4a7cb260b564a94a1191: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal/wals/1453e833d0ad494583c44b5496088f6d
I20240502 07:59:55.544098 20537 consensus_queue.cc:237] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } }
I20240502 07:59:55.545676 22475 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.545785 20272 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.545787 22593 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.546205 20935 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.546216 24556 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.546476 24556 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.546694 24556 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.546907 24556 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.548350 19987 catalog_manager.cc:4830] TS b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331): tablet 1453e833d0ad494583c44b5496088f6d (table rebalance_test_table_1 [id=dd634821c68c405c9e104a7053619a61]) successfully deleted
I20240502 07:59:55.548866 20143 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.548866 20800 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.549381 24556 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.549572 24557 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.550647 25091 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.551366 20800 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.551575 22593 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.551661 20272 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.553555 20143 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.553967 19994 catalog_manager.cc:5484] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P c15e23a34059456788f9775031e27ec0 reported cstate change: config changed from index 9 to 10, VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) evicted. New cstate: current_term: 1 leader_uuid: "e951f009258e473cb57cf8773c7975d4" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.555467 22475 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.555953 20935 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.557790 19986 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet dbf4b90cac9e4ff3b2936a1fa2ca8561 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:55.558233 20386 tablet_service.cc:1514] Processing DeleteTablet for tablet dbf4b90cac9e4ff3b2936a1fa2ca8561 with delete_type TABLET_DATA_TOMBSTONED (TS 35642aa2099c40e59bc2092604078e07 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:43238
I20240502 07:59:55.559010 25097 tablet_replica.cc:330] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 35642aa2099c40e59bc2092604078e07: stopping tablet replica
I20240502 07:59:55.559173 25097 raft_consensus.cc:2229] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:55.559478 25097 raft_consensus.cc:2258] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:55.560691 25097 ts_tablet_manager.cc:1875] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 35642aa2099c40e59bc2092604078e07: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:55.562489 25097 ts_tablet_manager.cc:1888] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 35642aa2099c40e59bc2092604078e07: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240502 07:59:55.562551 25097 log.cc:1196] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 35642aa2099c40e59bc2092604078e07: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal/wals/dbf4b90cac9e4ff3b2936a1fa2ca8561
I20240502 07:59:55.563102 19988 catalog_manager.cc:4830] TS 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587): tablet dbf4b90cac9e4ff3b2936a1fa2ca8561 (table rebalance_test_table_2 [id=30a30b02a2404507a95fc3b5bb77bdd1]) successfully deleted
I20240502 07:59:55.591671 20668 consensus_queue.cc:237] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } }
I20240502 07:59:55.593545 20143 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:55.593585 20405 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:55.593597 20537 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:55.594098 20272 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:55.594074 24819 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:55.594233 25135 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:55.594323 22593 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
W20240502 07:59:55.594349 20606 consensus_peers.cc:485] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bbae35bae8c047a6982e128ffb77ca86. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:55.594504 24955 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:55.594655 25136 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:55.594852 20935 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:55.595820 25135 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:55.596446 25135 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:55.597118 25134 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:55.598098 20537 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:55.598544 20935 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:55.601243 20000 catalog_manager.cc:5484] T bbae35bae8c047a6982e128ffb77ca86 P e951f009258e473cb57cf8773c7975d4 reported cstate change: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New cstate: current_term: 1 leader_uuid: "b443e68b6212450286869124684bdfff" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:55.603672 20272 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:55.603669 20405 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:55.606175 22593 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:55.606770 20143 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:55.622483 24987 raft_consensus.cc:1050] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191: attempting to promote NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 to VOTER
I20240502 07:59:55.622973 24987 consensus_queue.cc:237] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } }
I20240502 07:59:55.624616 22593 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.624775 22741 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.624994 24842 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.625232 20143 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.625350 20935 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.625669 25160 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.625873 20537 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.625887 20800 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.626262 24842 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.626397 24632 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.626706 25162 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.626853 24632 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.626937 20405 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.628057 25160 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.630261 25161 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.630767 20800 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.631009 22593 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.632277 22741 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.633309 20537 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.633533 20001 catalog_manager.cc:5484] T ae6d597fbc9242539edee8685a015d99 P ff9f2ccccdfa48a1afbc0e371f41c662 reported cstate change: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.634666 20143 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.636335 20405 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.639936 20935 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.642940 20272 consensus_queue.cc:237] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } }
I20240502 07:59:55.644826 20405 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.644891 20537 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.644915 22593 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.645228 25160 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.645264 20935 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.645395 24987 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.645541 24842 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.645584 20800 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.645907 24987 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.646117 25160 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.647315 22741 raft_consensus.cc:1261] T ae6d597fbc9242539edee8685a015d99 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.647524 24987 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.647696 24842 consensus_queue.cc:1035] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.648478 20935 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.648787 22741 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.649062 19987 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet ae6d597fbc9242539edee8685a015d99 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:55.649485 20405 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.650718 20537 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.651880 20800 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.652056 20000 catalog_manager.cc:5484] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 reported cstate change: config changed from index 9 to 10, VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) evicted. New cstate: current_term: 1 leader_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:55.655982 22593 raft_consensus.cc:2932] T ae6d597fbc9242539edee8685a015d99 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.656229 20122 tablet_service.cc:1514] Processing DeleteTablet for tablet ae6d597fbc9242539edee8685a015d99 with delete_type TABLET_DATA_TOMBSTONED (TS 29a7180fe4574c09a8dce7c27500aab2 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:44498
I20240502 07:59:55.656642 25025 tablet_replica.cc:330] T ae6d597fbc9242539edee8685a015d99 P 29a7180fe4574c09a8dce7c27500aab2: stopping tablet replica
I20240502 07:59:55.656788 25025 raft_consensus.cc:2229] T ae6d597fbc9242539edee8685a015d99 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:55.656996 25025 raft_consensus.cc:2258] T ae6d597fbc9242539edee8685a015d99 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:55.662488 25025 ts_tablet_manager.cc:1875] T ae6d597fbc9242539edee8685a015d99 P 29a7180fe4574c09a8dce7c27500aab2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:55.664378 25025 ts_tablet_manager.cc:1888] T ae6d597fbc9242539edee8685a015d99 P 29a7180fe4574c09a8dce7c27500aab2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240502 07:59:55.664440 25025 log.cc:1196] T ae6d597fbc9242539edee8685a015d99 P 29a7180fe4574c09a8dce7c27500aab2: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal/wals/ae6d597fbc9242539edee8685a015d99
I20240502 07:59:55.664907 19986 catalog_manager.cc:4830] TS 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703): tablet ae6d597fbc9242539edee8685a015d99 (table rebalance_test_table_5 [id=ed17eb489c6e490e9427e387589e51fe]) successfully deleted
I20240502 07:59:55.673496 25073 raft_consensus.cc:1050] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8: attempting to promote NON_VOTER f014c4ad4c1342219a1004181ac33ed8 to VOTER
I20240502 07:59:55.674057 25073 consensus_queue.cc:237] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } }
I20240502 07:59:55.676100 20405 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.676216 22741 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.676354 20668 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.676615 22593 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.676734 20143 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.676744 25054 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.677093 25054 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.677403 20273 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.678011 25070 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.678328 25070 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.678619 25070 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.678941 25070 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.680471 20800 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.680851 25054 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.682833 25136 raft_consensus.cc:1050] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff: attempting to promote NON_VOTER f014c4ad4c1342219a1004181ac33ed8 to VOTER
I20240502 07:59:55.682722 25054 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.683368 25136 consensus_queue.cc:237] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } }
I20240502 07:59:55.684540 20800 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.685292 20406 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.685298 20143 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.685688 20405 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.686131 22740 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.686430 22593 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.686621 25134 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.686949 20801 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.687739 24819 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.688058 24819 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.688218 24955 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.688442 19994 catalog_manager.cc:5484] T 15bb9480e88b41219cc91168cc438c70 P 35642aa2099c40e59bc2092604078e07 reported cstate change: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b5f471f8b5944986a7d878cf6ac550b8" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.688813 20935 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.688789 25201 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.689431 24955 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.690009 20272 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.691772 24819 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.693907 22593 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.694859 20143 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.696132 25134 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.697969 20668 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.699782 22741 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.699940 22593 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.700703 20405 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.700788 20801 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.702387 20273 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.703663 19999 catalog_manager.cc:5484] T 1bc5937510214c3da7731d3ad2a67842 P c15e23a34059456788f9775031e27ec0 reported cstate change: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b443e68b6212450286869124684bdfff" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.705876 20935 consensus_queue.cc:237] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } }
I20240502 07:59:55.708161 20801 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.708271 22593 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.708367 20142 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.708570 25056 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.708796 25054 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.709298 20143 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.709339 22741 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.709659 25054 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.710299 25056 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.710414 20405 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.712503 25054 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.713191 20668 raft_consensus.cc:1261] T 15bb9480e88b41219cc91168cc438c70 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.714351 20935 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.716092 24618 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.716707 20668 consensus_queue.cc:237] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } }
I20240502 07:59:55.716971 22741 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.718318 24650 raft_consensus.cc:1050] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2: attempting to promote NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 to VOTER
I20240502 07:59:55.718466 25074 consensus_queue.cc:1035] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.718641 22740 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.718850 24650 consensus_queue.cc:237] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } }
I20240502 07:59:55.720492 19988 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 15bb9480e88b41219cc91168cc438c70 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:55.720635 20935 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.721014 25199 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.721513 20272 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.721596 20273 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.721616 20406 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.721962 20537 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.721999 24956 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.722468 22593 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.722779 24956 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.723076 22609 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.723786 22609 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.724225 22741 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.724670 20000 catalog_manager.cc:5484] T 15bb9480e88b41219cc91168cc438c70 P 35642aa2099c40e59bc2092604078e07 reported cstate change: config changed from index 9 to 10, VOTER b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) evicted. New cstate: current_term: 1 leader_uuid: "b5f471f8b5944986a7d878cf6ac550b8" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.724956 20671 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.725117 24956 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.725421 24956 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.725442 20405 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.725517 25134 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.724972 20668 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.728154 20801 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.728865 20143 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.725791 25200 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
I20240502 07:59:55.731488 25134 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.731662 25199 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.733094 20253 tablet_service.cc:1514] Processing DeleteTablet for tablet 15bb9480e88b41219cc91168cc438c70 with delete_type TABLET_DATA_TOMBSTONED (TS b7604c0eb3bf4a7cb260b564a94a1191 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:59920
I20240502 07:59:55.733299 25102 tablet_replica.cc:330] T 15bb9480e88b41219cc91168cc438c70 P b7604c0eb3bf4a7cb260b564a94a1191: stopping tablet replica
I20240502 07:59:55.733425 25102 raft_consensus.cc:2229] T 15bb9480e88b41219cc91168cc438c70 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:55.733726 25201 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.735320 22740 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.735757 25102 raft_consensus.cc:2258] T 15bb9480e88b41219cc91168cc438c70 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:55.735731 24956 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.736168 24819 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.736502 25102 ts_tablet_manager.cc:1875] T 15bb9480e88b41219cc91168cc438c70 P b7604c0eb3bf4a7cb260b564a94a1191: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:55.737365 20935 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.738030 24956 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.738497 19987 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 1bc5937510214c3da7731d3ad2a67842 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:55.738641 25102 ts_tablet_manager.cc:1888] T 15bb9480e88b41219cc91168cc438c70 P b7604c0eb3bf4a7cb260b564a94a1191: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240502 07:59:55.738713 25102 log.cc:1196] T 15bb9480e88b41219cc91168cc438c70 P b7604c0eb3bf4a7cb260b564a94a1191: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal/wals/15bb9480e88b41219cc91168cc438c70
I20240502 07:59:55.738804 20142 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.739377 20933 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.739382 20801 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.739992 19999 catalog_manager.cc:5484] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff reported cstate change: config changed from index 9 to 10, VOTER b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) evicted. New cstate: current_term: 1 leader_uuid: "b443e68b6212450286869124684bdfff" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:55.741708 19987 catalog_manager.cc:4830] TS b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331): tablet 15bb9480e88b41219cc91168cc438c70 (table rebalance_test_table_0 [id=89c0cb0099e94f1aa93c3eb299e446f0]) successfully deleted
I20240502 07:59:55.742511 22609 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.743188 22740 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.744161 20405 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.744885 20253 tablet_service.cc:1514] Processing DeleteTablet for tablet 1bc5937510214c3da7731d3ad2a67842 with delete_type TABLET_DATA_TOMBSTONED (TS b7604c0eb3bf4a7cb260b564a94a1191 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:59920
I20240502 07:59:55.744572 25221 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.746254 20406 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.746551 20143 raft_consensus.cc:2932] T 15bb9480e88b41219cc91168cc438c70 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.748697 20801 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.749421 25102 tablet_replica.cc:330] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191: stopping tablet replica
I20240502 07:59:55.750147 25102 raft_consensus.cc:2229] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:55.750078 25220 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.751821 25102 raft_consensus.cc:2258] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:55.752115 22609 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.752166 20273 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.752377 20671 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.752996 20935 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.754009 25102 ts_tablet_manager.cc:1875] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:55.754828 19999 catalog_manager.cc:5484] T 9ad40762dc784dfa9f3507e025db32b8 P ff9f2ccccdfa48a1afbc0e371f41c662 reported cstate change: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "29a7180fe4574c09a8dce7c27500aab2" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.755622 22741 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.757567 20537 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.757892 25102 ts_tablet_manager.cc:1888] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240502 07:59:55.757972 25102 log.cc:1196] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal/wals/1bc5937510214c3da7731d3ad2a67842
I20240502 07:59:55.758885 19987 catalog_manager.cc:4830] TS b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331): tablet 1bc5937510214c3da7731d3ad2a67842 (table rebalance_test_table_1 [id=dd634821c68c405c9e104a7053619a61]) successfully deleted
I20240502 07:59:55.761565 20405 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.764993 20143 consensus_queue.cc:237] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } }
I20240502 07:59:55.766786 22609 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.766865 20935 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.766906 22741 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.767086 20537 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.767261 20273 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.767362 25219 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.767874 20671 raft_consensus.cc:1261] T 9ad40762dc784dfa9f3507e025db32b8 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.767853 25223 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.768220 24650 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.768543 24650 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.768698 25222 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.769896 25223 consensus_queue.cc:1035] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.770978 25224 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.771765 20671 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.771975 22741 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.772243 20935 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.772653 20273 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.775354 19992 catalog_manager.cc:5484] T 9ad40762dc784dfa9f3507e025db32b8 P b7604c0eb3bf4a7cb260b564a94a1191 reported cstate change: config changed from index 9 to 10, VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) evicted. New cstate: current_term: 1 leader_uuid: "29a7180fe4574c09a8dce7c27500aab2" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.776610 19986 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 9ad40762dc784dfa9f3507e025db32b8 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:55.777628 22609 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.778638 20386 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ad40762dc784dfa9f3507e025db32b8 with delete_type TABLET_DATA_TOMBSTONED (TS 35642aa2099c40e59bc2092604078e07 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:43238
I20240502 07:59:55.778968 25097 tablet_replica.cc:330] T 9ad40762dc784dfa9f3507e025db32b8 P 35642aa2099c40e59bc2092604078e07: stopping tablet replica
I20240502 07:59:55.779500 20537 raft_consensus.cc:2932] T 9ad40762dc784dfa9f3507e025db32b8 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:55.779944 25097 raft_consensus.cc:2229] T 9ad40762dc784dfa9f3507e025db32b8 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:55.780221 25097 raft_consensus.cc:2258] T 9ad40762dc784dfa9f3507e025db32b8 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:55.780979 25097 ts_tablet_manager.cc:1875] T 9ad40762dc784dfa9f3507e025db32b8 P 35642aa2099c40e59bc2092604078e07: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:55.784457 25162 raft_consensus.cc:1050] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191: attempting to promote NON_VOTER f014c4ad4c1342219a1004181ac33ed8 to VOTER
I20240502 07:59:55.784534 25097 ts_tablet_manager.cc:1888] T 9ad40762dc784dfa9f3507e025db32b8 P 35642aa2099c40e59bc2092604078e07: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240502 07:59:55.784610 25097 log.cc:1196] T 9ad40762dc784dfa9f3507e025db32b8 P 35642aa2099c40e59bc2092604078e07: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal/wals/9ad40762dc784dfa9f3507e025db32b8
I20240502 07:59:55.785020 25162 consensus_queue.cc:237] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } }
I20240502 07:59:55.785079 19988 catalog_manager.cc:4830] TS 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587): tablet 9ad40762dc784dfa9f3507e025db32b8 (table rebalance_test_table_6 [id=db1e20e82b894565a302e53bd7791607]) successfully deleted
I20240502 07:59:55.786688 20935 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.786936 22475 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.787094 24842 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.787227 20537 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.787674 22741 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.787910 24987 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.788409 24632 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.788583 20405 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.788959 24842 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.789103 20671 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.789341 20143 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:55.789692 24842 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.789849 24632 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.790402 24632 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:55.792084 24632 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.792680 20143 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.792718 20671 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.794149 22741 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.794749 22475 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.795038 19999 catalog_manager.cc:5484] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 reported cstate change: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:55.795573 20537 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.796983 20935 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.802248 20405 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.804729 20273 consensus_queue.cc:237] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } }
I20240502 07:59:55.806210 22741 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.806313 20935 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.806389 20537 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.806484 22475 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.806569 25161 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.806733 24987 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.806865 24842 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.806973 25162 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.807715 20671 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.808099 24842 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.809317 20405 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:55.809665 25162 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:55.809633 24987 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.810539 20405 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.811448 20537 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.811476 20671 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.812059 20935 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.812588 19987 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 0e2908de10684c84af542f16ff8727cb with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:55.813290 20001 catalog_manager.cc:5484] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 reported cstate change: config changed from index 9 to 10, VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) evicted. New cstate: current_term: 1 leader_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:55.814637 22475 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.815765 22741 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:55.817178 20122 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e2908de10684c84af542f16ff8727cb with delete_type TABLET_DATA_TOMBSTONED (TS 29a7180fe4574c09a8dce7c27500aab2 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:44498
I20240502 07:59:55.817418 25025 tablet_replica.cc:330] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2: stopping tablet replica
I20240502 07:59:55.817579 25025 raft_consensus.cc:2229] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:55.817845 25025 raft_consensus.cc:2258] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:55.819664 25025 ts_tablet_manager.cc:1875] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:55.821420 25025 ts_tablet_manager.cc:1888] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240502 07:59:55.821465 25025 log.cc:1196] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal/wals/0e2908de10684c84af542f16ff8727cb
I20240502 07:59:55.821853 19986 catalog_manager.cc:4830] TS 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703): tablet 0e2908de10684c84af542f16ff8727cb (table rebalance_test_table_4 [id=8edc4db66dcd46ee8429441e0ce671a3]) successfully deleted
I20240502 07:59:56.003039 25281 ts_tablet_manager.cc:902] T bbae35bae8c047a6982e128ffb77ca86 P f014c4ad4c1342219a1004181ac33ed8: Initiating tablet copy from peer b443e68b6212450286869124684bdfff (127.3.139.69:37923)
I20240502 07:59:56.003661 25281 tablet_copy_client.cc:308] T bbae35bae8c047a6982e128ffb77ca86 P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.69:37923
I20240502 07:59:56.004145 20691 tablet_copy_service.cc:140] P b443e68b6212450286869124684bdfff: Received BeginTabletCopySession request for tablet bbae35bae8c047a6982e128ffb77ca86 from peer f014c4ad4c1342219a1004181ac33ed8 ({username='slave'} at 127.3.139.72:49807)
I20240502 07:59:56.004251 20691 tablet_copy_service.cc:161] P b443e68b6212450286869124684bdfff: Beginning new tablet copy session on tablet bbae35bae8c047a6982e128ffb77ca86 from peer f014c4ad4c1342219a1004181ac33ed8 at {username='slave'} at 127.3.139.72:49807: session id = f014c4ad4c1342219a1004181ac33ed8-bbae35bae8c047a6982e128ffb77ca86
I20240502 07:59:56.004961 20691 tablet_copy_source_session.cc:205] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:56.005844 25281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbae35bae8c047a6982e128ffb77ca86. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:56.007843 25281 tablet_copy_client.cc:783] T bbae35bae8c047a6982e128ffb77ca86 P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:56.007977 25281 tablet_copy_client.cc:647] T bbae35bae8c047a6982e128ffb77ca86 P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:56.009059 25281 tablet_copy_client.cc:523] T bbae35bae8c047a6982e128ffb77ca86 P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:56.010017 25281 tablet_bootstrap.cc:492] T bbae35bae8c047a6982e128ffb77ca86 P f014c4ad4c1342219a1004181ac33ed8: Bootstrap starting.
I20240502 07:59:56.028632 25281 tablet_bootstrap.cc:492] T bbae35bae8c047a6982e128ffb77ca86 P f014c4ad4c1342219a1004181ac33ed8: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:56.028865 25281 tablet_bootstrap.cc:492] T bbae35bae8c047a6982e128ffb77ca86 P f014c4ad4c1342219a1004181ac33ed8: Bootstrap complete.
I20240502 07:59:56.029027 25281 ts_tablet_manager.cc:1367] T bbae35bae8c047a6982e128ffb77ca86 P f014c4ad4c1342219a1004181ac33ed8: Time spent bootstrapping tablet: real 0.019s user 0.018s sys 0.000s
I20240502 07:59:56.029385 25281 raft_consensus.cc:348] T bbae35bae8c047a6982e128ffb77ca86 P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } }
I20240502 07:59:56.029500 25281 raft_consensus.cc:729] T bbae35bae8c047a6982e128ffb77ca86 P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f014c4ad4c1342219a1004181ac33ed8, State: Initialized, Role: LEARNER
I20240502 07:59:56.029594 25281 consensus_queue.cc:260] T bbae35bae8c047a6982e128ffb77ca86 P f014c4ad4c1342219a1004181ac33ed8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } }
I20240502 07:59:56.029850 25281 ts_tablet_manager.cc:1398] T bbae35bae8c047a6982e128ffb77ca86 P f014c4ad4c1342219a1004181ac33ed8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:56.030468 20691 tablet_copy_service.cc:342] P b443e68b6212450286869124684bdfff: Request end of tablet copy session f014c4ad4c1342219a1004181ac33ed8-bbae35bae8c047a6982e128ffb77ca86 received from {username='slave'} at 127.3.139.72:49807
I20240502 07:59:56.030561 20691 tablet_copy_service.cc:433] P b443e68b6212450286869124684bdfff: ending tablet copy session f014c4ad4c1342219a1004181ac33ed8-bbae35bae8c047a6982e128ffb77ca86 on tablet bbae35bae8c047a6982e128ffb77ca86 with peer f014c4ad4c1342219a1004181ac33ed8
I20240502 07:59:56.490458 22741 raft_consensus.cc:1203] T bbae35bae8c047a6982e128ffb77ca86 P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240502 07:59:56.957909 25200 raft_consensus.cc:1050] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff: attempting to promote NON_VOTER f014c4ad4c1342219a1004181ac33ed8 to VOTER
I20240502 07:59:56.958494 25200 consensus_queue.cc:237] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } }
I20240502 07:59:56.960422 20143 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:56.960551 22740 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:56.960610 22609 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:56.961088 20272 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:56.961441 20405 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:56.961462 20935 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:56.961578 25201 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:56.961934 25201 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:56.962036 20537 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:56.962255 25201 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:56.962524 25297 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:56.962677 25201 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:56.963356 25200 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:56.963500 25297 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:56.965668 25200 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:56.967761 20272 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:56.967761 20935 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:56.968370 19999 catalog_manager.cc:5484] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff reported cstate change: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b443e68b6212450286869124684bdfff" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:56.967037 20537 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:56.969794 20143 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:56.971709 22609 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:56.973064 20405 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:56.974046 22740 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:56.980226 20671 consensus_queue.cc:237] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } }
I20240502 07:59:56.981606 22740 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:56.981626 22609 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:56.982029 20143 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:56.982106 25201 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:56.982295 20935 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:56.982338 20537 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:56.982426 25200 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:56.982626 20272 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:56.982677 25201 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:56.982971 25318 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:56.983177 25201 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:56.984372 25318 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:56.985200 25297 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:56.986517 20272 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:56.987305 20935 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:56.989089 20000 catalog_manager.cc:5484] T bbae35bae8c047a6982e128ffb77ca86 P b7604c0eb3bf4a7cb260b564a94a1191 reported cstate change: config changed from index 9 to 10, VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) evicted. New cstate: current_term: 1 leader_uuid: "b443e68b6212450286869124684bdfff" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:56.989353 19987 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet bbae35bae8c047a6982e128ffb77ca86 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:56.989346 22609 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:56.991626 20386 tablet_service.cc:1514] Processing DeleteTablet for tablet bbae35bae8c047a6982e128ffb77ca86 with delete_type TABLET_DATA_TOMBSTONED (TS 35642aa2099c40e59bc2092604078e07 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:43238
I20240502 07:59:56.991935 20537 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:56.993552 22740 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:56.991612 20143 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:56.997030 25332 tablet_replica.cc:330] T bbae35bae8c047a6982e128ffb77ca86 P 35642aa2099c40e59bc2092604078e07: stopping tablet replica
I20240502 07:59:56.997207 25332 raft_consensus.cc:2229] T bbae35bae8c047a6982e128ffb77ca86 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:56.997465 25332 raft_consensus.cc:2258] T bbae35bae8c047a6982e128ffb77ca86 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:56.998070 25332 ts_tablet_manager.cc:1875] T bbae35bae8c047a6982e128ffb77ca86 P 35642aa2099c40e59bc2092604078e07: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:56.999941 25332 ts_tablet_manager.cc:1888] T bbae35bae8c047a6982e128ffb77ca86 P 35642aa2099c40e59bc2092604078e07: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240502 07:59:57.000005 25332 log.cc:1196] T bbae35bae8c047a6982e128ffb77ca86 P 35642aa2099c40e59bc2092604078e07: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal/wals/bbae35bae8c047a6982e128ffb77ca86
I20240502 07:59:57.000420 19988 catalog_manager.cc:4830] TS 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587): tablet bbae35bae8c047a6982e128ffb77ca86 (table rebalance_test_table_3 [id=1388635dd26f4d3791ee92dfbf8d374a]) successfully deleted
I20240502 07:59:57.146364 20386 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:59:57.147241 20122 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:59:57.151960 20253 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:59:57.156438 22456 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:57.164813 20517 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:59:57.169132 20649 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:59:57.176946 20781 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:59:57.190189 20915 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:59:57.202498 22588 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:57.209630 22721 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:57.311929 20386 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:59:57.312078 20649 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:59:57.318010 20253 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:59:57.319149 20915 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:59:57.322647 22456 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:57.329361 20122 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:59:57.337446 22588 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:57.339689 22721 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:57.349895 20517 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:59:57.362387 20781 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:59:57.422475 20935 consensus_queue.cc:237] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: NON_VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: true } }
W20240502 07:59:57.425712 20871 consensus_peers.cc:485] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587): Couldn't send request to peer 35642aa2099c40e59bc2092604078e07. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:57.426285 20272 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 07:59:57.426566 20801 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 07:59:57.427176 20537 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 07:59:57.427240 22475 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 07:59:57.427600 20143 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 07:59:57.426842 25459 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 07:59:57.427839 24618 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 07:59:57.428036 25315 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 07:59:57.428352 22740 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 07:59:57.428371 25315 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 07:59:57.428921 24618 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 07:59:57.430447 25462 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 07:59:57.434211 20405 consensus_queue.cc:237] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } }
I20240502 07:59:57.434484 20143 consensus_queue.cc:237] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: true } }
I20240502 07:59:57.434542 22475 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: NON_VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: true } } }
I20240502 07:59:57.436625 20935 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:57.436833 20933 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:57.436822 25462 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: NON_VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: true } } }
I20240502 07:59:57.437220 25009 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:57.437253 20142 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: NON_VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: true } } }
I20240502 07:59:57.437903 20671 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:57.438464 22609 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:57.438783 20801 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:57.438935 20405 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:57.439302 22740 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:57.439664 20272 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:57.439613 22741 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: NON_VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: true } } }
I20240502 07:59:57.441238 20800 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:57.442385 20539 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:57.442834 19993 catalog_manager.cc:5484] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P bfb1b3d596ef474faa229a835329cf20 reported cstate change: config changed from index 10 to 11, NON_VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) added. New cstate: current_term: 1 leader_uuid: "b5f471f8b5944986a7d878cf6ac550b8" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: NON_VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: true } } }
W20240502 07:59:57.443064 20342 consensus_peers.cc:485] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 574fdd7a3c114ac9928ac12d1a7a4186. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:57.443171 25009 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:57.443460 25009 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:57.443533 25224 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:57.443936 25224 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:57.444247 25224 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:57.444521 25224 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:57.444561 20273 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: NON_VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: true } } }
I20240502 07:59:57.445952 20272 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:57.446226 25224 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20240502 07:59:57.446399 20078 consensus_peers.cc:485] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 -> Peer b443e68b6212450286869124684bdfff (127.3.139.69:37923): Couldn't send request to peer b443e68b6212450286869124684bdfff. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:57.446542 25224 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:57.447875 20801 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: NON_VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: true } } }
I20240502 07:59:57.451136 25314 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:57.451442 25314 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:57.453068 20537 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: NON_VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: true } } }
I20240502 07:59:57.453316 25471 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER b443e68b6212450286869124684bdfff (127.3.139.69) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: true } } }
I20240502 07:59:57.453722 20933 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER b443e68b6212450286869124684bdfff (127.3.139.69) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: true } } }
I20240502 07:59:57.456071 20273 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER b443e68b6212450286869124684bdfff (127.3.139.69) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: true } } }
I20240502 07:59:57.458477 20143 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:57.458914 25009 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:57.458891 19997 catalog_manager.cc:5484] T 880876d792554606a7ccf21f9ec08e0f P b5f471f8b5944986a7d878cf6ac550b8 reported cstate change: config changed from index 7 to 8, NON_VOTER b443e68b6212450286869124684bdfff (127.3.139.69) added. New cstate: current_term: 1 leader_uuid: "29a7180fe4574c09a8dce7c27500aab2" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: true } } }
I20240502 07:59:57.459276 25314 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:57.460012 20143 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:57.461391 20933 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:57.461695 20668 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:57.463670 22741 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:57.464810 19994 catalog_manager.cc:5484] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 reported cstate change: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New cstate: current_term: 1 leader_uuid: "35642aa2099c40e59bc2092604078e07" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:57.468510 20539 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER b443e68b6212450286869124684bdfff (127.3.139.69) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: true } } }
I20240502 07:59:57.469571 20272 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:57.472041 22593 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER b443e68b6212450286869124684bdfff (127.3.139.69) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: true } } }
I20240502 07:59:57.475628 20801 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:57.477751 20801 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER b443e68b6212450286869124684bdfff (127.3.139.69) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: true } } }
I20240502 07:59:57.482096 20405 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER b443e68b6212450286869124684bdfff (127.3.139.69) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: true } } }
I20240502 07:59:57.486572 20668 consensus_queue.cc:237] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: NON_VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { promote: true } }
I20240502 07:59:57.488775 20405 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 07:59:57.488871 20143 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 07:59:57.489315 22741 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 07:59:57.489658 22593 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 07:59:57.489980 20539 consensus_queue.cc:237] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } }
I20240502 07:59:57.490036 25512 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 07:59:57.490664 20801 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 07:59:57.490851 25319 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 07:59:57.491200 25319 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20240502 07:59:57.491932 20606 consensus_peers.cc:485] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff -> Peer b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331): Couldn't send request to peer b7604c0eb3bf4a7cb260b564a94a1191. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:57.492002 20933 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 07:59:57.492435 25297 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 07:59:57.492727 25297 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 07:59:57.493584 20801 consensus_queue.cc:237] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: NON_VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { promote: true } }
I20240502 07:59:57.494342 20272 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:57.494580 22741 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:57.494657 20142 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:57.495309 20406 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:57.495301 25093 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:57.495995 25513 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 07:59:57.496170 20935 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:57.496455 20142 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:57.496487 25513 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: NON_VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { promote: true } } }
W20240502 07:59:57.496698 20474 consensus_peers.cc:485] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a973de06330c487ba54a8a36b1e31b54. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:57.496805 25093 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:57.496999 25528 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:57.497102 20933 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:57.497097 25093 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:57.497336 25093 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:57.497444 25334 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:57.497529 25027 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:57.498016 22593 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:57.498304 20668 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:57.498737 20272 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:57.498849 20801 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
W20240502 07:59:57.499219 20737 consensus_peers.cc:485] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 -> Peer e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315): Couldn't send request to peer e951f009258e473cb57cf8773c7975d4. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:57.499253 20406 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: NON_VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { promote: true } } }
I20240502 07:59:57.500154 20405 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:57.500523 25531 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:57.502147 19995 catalog_manager.cc:5484] T 1bc5937510214c3da7731d3ad2a67842 P 35642aa2099c40e59bc2092604078e07 reported cstate change: config changed from index 10 to 11, NON_VOTER b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) added. New cstate: current_term: 1 leader_uuid: "b443e68b6212450286869124684bdfff" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: NON_VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { promote: true } } }
I20240502 07:59:57.502730 25520 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:57.503321 20140 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:57.505067 20405 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:57.506467 19995 catalog_manager.cc:5484] T a973de06330c487ba54a8a36b1e31b54 P 29a7180fe4574c09a8dce7c27500aab2 reported cstate change: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New cstate: current_term: 1 leader_uuid: "e951f009258e473cb57cf8773c7975d4" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:57.506779 25530 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:57.507146 25530 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:57.507431 25530 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:57.511243 25518 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:57.511629 20272 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:57.511785 20934 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:57.512351 20800 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:57.516541 25530 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: NON_VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { promote: true } } }
I20240502 07:59:57.517052 20406 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: NON_VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { promote: true } } }
I20240502 07:59:57.520107 19995 catalog_manager.cc:5484] T 6a77bda800d44c4bbe1b082a5ea05b55 P 35642aa2099c40e59bc2092604078e07 reported cstate change: config changed from index 7 to 8, NON_VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) added. New cstate: current_term: 1 leader_uuid: "c15e23a34059456788f9775031e27ec0" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: NON_VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { promote: true } } }
I20240502 07:59:57.520375 20140 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: NON_VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { promote: true } } }
I20240502 07:59:57.522565 22740 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: NON_VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { promote: true } } }
I20240502 07:59:57.523710 20140 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: NON_VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { promote: true } } }
I20240502 07:59:57.523911 20800 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: NON_VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { promote: true } } }
I20240502 07:59:57.525085 22609 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: NON_VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { promote: true } } }
I20240502 07:59:57.532269 20407 consensus_queue.cc:237] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } }
W20240502 07:59:57.534375 20342 consensus_peers.cc:485] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e61fe3cb61c240f9b2c898aff1951144. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:57.534636 22609 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 07:59:57.535331 20272 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: NON_VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { promote: true } } }
I20240502 07:59:57.535668 25474 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 07:59:57.536612 20273 consensus_queue.cc:237] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: NON_VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { promote: true } }
I20240502 07:59:57.538743 22740 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 07:59:57.538879 22475 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 07:59:57.539208 25161 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 07:59:57.539446 20671 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 07:59:57.539562 25161 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 07:59:57.539727 20539 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
W20240502 07:59:57.539878 20209 consensus_peers.cc:485] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703): Couldn't send request to peer 29a7180fe4574c09a8dce7c27500aab2. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:57.540129 20934 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 07:59:57.540606 25161 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 07:59:57.540894 25161 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 07:59:57.541402 20800 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 07:59:57.541702 20539 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 07:59:57.541814 20406 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 07:59:57.542243 25161 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 07:59:57.542336 20932 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 07:59:57.542443 25480 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 07:59:57.542670 25472 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 07:59:57.544032 25479 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 07:59:57.544924 22476 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 07:59:57.545218 25472 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:57.547191 20932 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:57.547322 22609 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:57.549819 19994 catalog_manager.cc:5484] T e61fe3cb61c240f9b2c898aff1951144 P ff9f2ccccdfa48a1afbc0e371f41c662 reported cstate change: config changed from index 10 to 11, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New cstate: current_term: 1 leader_uuid: "35642aa2099c40e59bc2092604078e07" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:57.550899 25480 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 07:59:57.552189 22476 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:57.552243 20800 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:57.552294 20539 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:57.552750 20668 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: NON_VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { promote: true } } }
I20240502 07:59:57.555985 22593 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: NON_VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { promote: true } } }
I20240502 07:59:57.559403 25161 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 07:59:57.560031 20273 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 07:59:57.560081 20933 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER e951f009258e473cb57cf8773c7975d4 (127.3.139.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: NON_VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { promote: true } } }
I20240502 07:59:57.560603 25009 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 07:59:57.561657 20273 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 07:59:57.561965 20933 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: NON_VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { promote: true } } }
I20240502 07:59:57.565347 25161 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: NON_VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { promote: true } } }
I20240502 07:59:57.566365 22741 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 07:59:57.566576 20934 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: NON_VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { promote: true } } }
I20240502 07:59:57.567873 20537 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: NON_VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { promote: true } } }
I20240502 07:59:57.568444 22475 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: NON_VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { promote: true } } }
I20240502 07:59:57.569731 20668 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: NON_VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { promote: true } } }
I20240502 07:59:57.571303 19997 catalog_manager.cc:5484] T 0e2908de10684c84af542f16ff8727cb P b5f471f8b5944986a7d878cf6ac550b8 reported cstate change: config changed from index 10 to 11, NON_VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) added. New cstate: current_term: 1 leader_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: NON_VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { promote: true } } }
I20240502 07:59:57.576902 20406 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: NON_VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { promote: true } } }
I20240502 07:59:57.577533 22741 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: NON_VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { promote: true } } }
I20240502 07:59:57.581861 20537 consensus_queue.cc:237] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: true } }
I20240502 07:59:57.584084 22593 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 07:59:57.584151 20273 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 07:59:57.584682 25521 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 07:59:57.584735 20140 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
W20240502 07:59:57.584949 20474 consensus_peers.cc:485] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 -> Peer b443e68b6212450286869124684bdfff (127.3.139.69:37923): Couldn't send request to peer b443e68b6212450286869124684bdfff. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:57.585110 25334 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 07:59:57.585500 25334 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 07:59:57.585681 22475 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 07:59:57.586010 25334 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 07:59:57.587591 20934 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 07:59:57.587610 20800 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 07:59:57.587970 25334 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 07:59:57.588488 25521 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 07:59:57.589133 25521 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER b443e68b6212450286869124684bdfff (127.3.139.69) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: true } } }
I20240502 07:59:57.591145 22593 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER b443e68b6212450286869124684bdfff (127.3.139.69) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: true } } }
I20240502 07:59:57.591356 20934 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER b443e68b6212450286869124684bdfff (127.3.139.69) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: true } } }
I20240502 07:59:57.591847 20140 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER b443e68b6212450286869124684bdfff (127.3.139.69) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: true } } }
I20240502 07:59:57.592113 22475 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER b443e68b6212450286869124684bdfff (127.3.139.69) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: true } } }
I20240502 07:59:57.592911 20273 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER b443e68b6212450286869124684bdfff (127.3.139.69) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: true } } }
I20240502 07:59:57.593555 19993 catalog_manager.cc:5484] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 reported cstate change: config changed from index 10 to 11, NON_VOTER b443e68b6212450286869124684bdfff (127.3.139.69) added. New cstate: current_term: 1 leader_uuid: "e951f009258e473cb57cf8773c7975d4" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:57.596068 20800 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER b443e68b6212450286869124684bdfff (127.3.139.69) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: true } } }
I20240502 07:59:57.606863 20671 consensus_queue.cc:237] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { promote: true } }
I20240502 07:59:57.608723 22593 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 07:59:57.608732 22741 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
W20240502 07:59:57.608879 20605 consensus_peers.cc:485] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff -> Peer c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085): Couldn't send request to peer c15e23a34059456788f9775031e27ec0. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:57.609724 20934 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 07:59:57.609733 20537 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 07:59:57.609733 20140 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 07:59:57.610112 25512 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 07:59:57.610287 25297 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 07:59:57.610571 20273 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 07:59:57.610574 25297 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 07:59:57.610911 25297 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 07:59:57.611624 25512 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 07:59:57.613253 25512 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 07:59:57.613888 25514 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { promote: true } } }
I20240502 07:59:57.614377 22593 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { promote: true } } }
I20240502 07:59:57.616209 20934 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { promote: true } } }
I20240502 07:59:57.616951 20273 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { promote: true } } }
I20240502 07:59:57.617714 20537 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { promote: true } } }
I20240502 07:59:57.618347 19992 catalog_manager.cc:5484] T bbae35bae8c047a6982e128ffb77ca86 P ff9f2ccccdfa48a1afbc0e371f41c662 reported cstate change: config changed from index 10 to 11, NON_VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) added. New cstate: current_term: 1 leader_uuid: "b443e68b6212450286869124684bdfff" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { promote: true } } }
I20240502 07:59:57.621448 22741 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { promote: true } } }
I20240502 07:59:57.621474 20140 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { promote: true } } }
I20240502 07:59:57.635654 20537 consensus_queue.cc:237] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: true } }
I20240502 07:59:57.637403 20406 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:57.637496 22741 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:57.637820 25522 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20240502 07:59:57.638037 20474 consensus_peers.cc:485] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 -> Peer b443e68b6212450286869124684bdfff (127.3.139.69:37923): Couldn't send request to peer b443e68b6212450286869124684bdfff. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:57.638545 25334 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:57.639513 20800 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:57.639637 20140 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:57.639933 25519 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:57.640295 25519 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:57.640348 20273 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:57.642056 25520 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:57.642349 25334 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER b443e68b6212450286869124684bdfff (127.3.139.69) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: true } } }
I20240502 07:59:57.643215 20934 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 07:59:57.644287 25518 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:57.644655 20406 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER b443e68b6212450286869124684bdfff (127.3.139.69) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: true } } }
I20240502 07:59:57.644687 20140 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER b443e68b6212450286869124684bdfff (127.3.139.69) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: true } } }
I20240502 07:59:57.646543 19992 catalog_manager.cc:5484] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 reported cstate change: config changed from index 7 to 8, NON_VOTER b443e68b6212450286869124684bdfff (127.3.139.69) added. New cstate: current_term: 1 leader_uuid: "e951f009258e473cb57cf8773c7975d4" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:57.647413 20273 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER b443e68b6212450286869124684bdfff (127.3.139.69) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: true } } }
I20240502 07:59:57.647589 22741 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER b443e68b6212450286869124684bdfff (127.3.139.69) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: true } } }
I20240502 07:59:57.648065 20934 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER b443e68b6212450286869124684bdfff (127.3.139.69) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: true } } }
I20240502 07:59:57.650949 20800 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER b443e68b6212450286869124684bdfff (127.3.139.69) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: true } } }
I20240502 07:59:57.891875 25700 ts_tablet_manager.cc:902] T a973de06330c487ba54a8a36b1e31b54 P ff9f2ccccdfa48a1afbc0e371f41c662: Initiating tablet copy from peer e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315)
I20240502 07:59:57.892362 25700 tablet_copy_client.cc:308] T a973de06330c487ba54a8a36b1e31b54 P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.68:40315
I20240502 07:59:57.892863 20559 tablet_copy_service.cc:140] P e951f009258e473cb57cf8773c7975d4: Received BeginTabletCopySession request for tablet a973de06330c487ba54a8a36b1e31b54 from peer ff9f2ccccdfa48a1afbc0e371f41c662 ({username='slave'} at 127.3.139.74:34453)
I20240502 07:59:57.892966 20559 tablet_copy_service.cc:161] P e951f009258e473cb57cf8773c7975d4: Beginning new tablet copy session on tablet a973de06330c487ba54a8a36b1e31b54 from peer ff9f2ccccdfa48a1afbc0e371f41c662 at {username='slave'} at 127.3.139.74:34453: session id = ff9f2ccccdfa48a1afbc0e371f41c662-a973de06330c487ba54a8a36b1e31b54
I20240502 07:59:57.893721 20559 tablet_copy_source_session.cc:205] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:57.894773 25700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a973de06330c487ba54a8a36b1e31b54. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:57.897184 25700 tablet_copy_client.cc:783] T a973de06330c487ba54a8a36b1e31b54 P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:57.897341 25700 tablet_copy_client.cc:647] T a973de06330c487ba54a8a36b1e31b54 P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:57.898365 25700 tablet_copy_client.cc:523] T a973de06330c487ba54a8a36b1e31b54 P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:57.899371 25700 tablet_bootstrap.cc:492] T a973de06330c487ba54a8a36b1e31b54 P ff9f2ccccdfa48a1afbc0e371f41c662: Bootstrap starting.
I20240502 07:59:57.921988 25700 tablet_bootstrap.cc:492] T a973de06330c487ba54a8a36b1e31b54 P ff9f2ccccdfa48a1afbc0e371f41c662: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:57.922312 25700 tablet_bootstrap.cc:492] T a973de06330c487ba54a8a36b1e31b54 P ff9f2ccccdfa48a1afbc0e371f41c662: Bootstrap complete.
I20240502 07:59:57.922475 25700 ts_tablet_manager.cc:1367] T a973de06330c487ba54a8a36b1e31b54 P ff9f2ccccdfa48a1afbc0e371f41c662: Time spent bootstrapping tablet: real 0.023s user 0.020s sys 0.000s
I20240502 07:59:57.922870 25700 raft_consensus.cc:348] T a973de06330c487ba54a8a36b1e31b54 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } }
I20240502 07:59:57.922996 25700 raft_consensus.cc:729] T a973de06330c487ba54a8a36b1e31b54 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ff9f2ccccdfa48a1afbc0e371f41c662, State: Initialized, Role: LEARNER
I20240502 07:59:57.923110 25700 consensus_queue.cc:260] T a973de06330c487ba54a8a36b1e31b54 P ff9f2ccccdfa48a1afbc0e371f41c662 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } }
I20240502 07:59:57.923395 25700 ts_tablet_manager.cc:1398] T a973de06330c487ba54a8a36b1e31b54 P ff9f2ccccdfa48a1afbc0e371f41c662: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240502 07:59:57.923897 20559 tablet_copy_service.cc:342] P e951f009258e473cb57cf8773c7975d4: Request end of tablet copy session ff9f2ccccdfa48a1afbc0e371f41c662-a973de06330c487ba54a8a36b1e31b54 received from {username='slave'} at 127.3.139.74:34453
I20240502 07:59:57.923990 20559 tablet_copy_service.cc:433] P e951f009258e473cb57cf8773c7975d4: ending tablet copy session ff9f2ccccdfa48a1afbc0e371f41c662-a973de06330c487ba54a8a36b1e31b54 on tablet a973de06330c487ba54a8a36b1e31b54 with peer ff9f2ccccdfa48a1afbc0e371f41c662
I20240502 07:59:57.949074 25700 ts_tablet_manager.cc:902] T 574fdd7a3c114ac9928ac12d1a7a4186 P ff9f2ccccdfa48a1afbc0e371f41c662: Initiating tablet copy from peer 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587)
I20240502 07:59:57.949610 25700 tablet_copy_client.cc:308] T 574fdd7a3c114ac9928ac12d1a7a4186 P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.67:45587
I20240502 07:59:57.950119 20427 tablet_copy_service.cc:140] P 35642aa2099c40e59bc2092604078e07: Received BeginTabletCopySession request for tablet 574fdd7a3c114ac9928ac12d1a7a4186 from peer ff9f2ccccdfa48a1afbc0e371f41c662 ({username='slave'} at 127.3.139.74:34321)
I20240502 07:59:57.950212 20427 tablet_copy_service.cc:161] P 35642aa2099c40e59bc2092604078e07: Beginning new tablet copy session on tablet 574fdd7a3c114ac9928ac12d1a7a4186 from peer ff9f2ccccdfa48a1afbc0e371f41c662 at {username='slave'} at 127.3.139.74:34321: session id = ff9f2ccccdfa48a1afbc0e371f41c662-574fdd7a3c114ac9928ac12d1a7a4186
I20240502 07:59:57.950966 20427 tablet_copy_source_session.cc:205] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:57.951913 25700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 574fdd7a3c114ac9928ac12d1a7a4186. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:57.953624 25700 tablet_copy_client.cc:783] T 574fdd7a3c114ac9928ac12d1a7a4186 P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:57.953761 25700 tablet_copy_client.cc:647] T 574fdd7a3c114ac9928ac12d1a7a4186 P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:57.955075 25700 tablet_copy_client.cc:523] T 574fdd7a3c114ac9928ac12d1a7a4186 P ff9f2ccccdfa48a1afbc0e371f41c662: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:57.956084 25700 tablet_bootstrap.cc:492] T 574fdd7a3c114ac9928ac12d1a7a4186 P ff9f2ccccdfa48a1afbc0e371f41c662: Bootstrap starting.
I20240502 07:59:57.963770 25726 ts_tablet_manager.cc:902] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b443e68b6212450286869124684bdfff: Initiating tablet copy from peer e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315)
I20240502 07:59:57.964517 25726 tablet_copy_client.cc:308] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b443e68b6212450286869124684bdfff: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.68:40315
I20240502 07:59:57.965087 20559 tablet_copy_service.cc:140] P e951f009258e473cb57cf8773c7975d4: Received BeginTabletCopySession request for tablet dbf4b90cac9e4ff3b2936a1fa2ca8561 from peer b443e68b6212450286869124684bdfff ({username='slave'} at 127.3.139.69:33433)
I20240502 07:59:57.965180 20559 tablet_copy_service.cc:161] P e951f009258e473cb57cf8773c7975d4: Beginning new tablet copy session on tablet dbf4b90cac9e4ff3b2936a1fa2ca8561 from peer b443e68b6212450286869124684bdfff at {username='slave'} at 127.3.139.69:33433: session id = b443e68b6212450286869124684bdfff-dbf4b90cac9e4ff3b2936a1fa2ca8561
I20240502 07:59:57.966076 20559 tablet_copy_source_session.cc:205] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:57.966579 25731 ts_tablet_manager.cc:902] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191: Initiating tablet copy from peer b443e68b6212450286869124684bdfff (127.3.139.69:37923)
I20240502 07:59:57.967099 25726 ts_tablet_manager.cc:1875] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b443e68b6212450286869124684bdfff: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240502 07:59:57.967307 25731 tablet_copy_client.cc:308] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.69:37923
I20240502 07:59:57.967824 20691 tablet_copy_service.cc:140] P b443e68b6212450286869124684bdfff: Received BeginTabletCopySession request for tablet 1bc5937510214c3da7731d3ad2a67842 from peer b7604c0eb3bf4a7cb260b564a94a1191 ({username='slave'} at 127.3.139.66:48499)
I20240502 07:59:57.967916 20691 tablet_copy_service.cc:161] P b443e68b6212450286869124684bdfff: Beginning new tablet copy session on tablet 1bc5937510214c3da7731d3ad2a67842 from peer b7604c0eb3bf4a7cb260b564a94a1191 at {username='slave'} at 127.3.139.66:48499: session id = b7604c0eb3bf4a7cb260b564a94a1191-1bc5937510214c3da7731d3ad2a67842
I20240502 07:59:57.968546 20691 tablet_copy_source_session.cc:205] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:57.969280 25726 ts_tablet_manager.cc:1888] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b443e68b6212450286869124684bdfff: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.6
I20240502 07:59:57.969398 25731 ts_tablet_manager.cc:1875] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240502 07:59:57.969450 25726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbf4b90cac9e4ff3b2936a1fa2ca8561. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:57.970609 25726 tablet_copy_client.cc:783] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b443e68b6212450286869124684bdfff: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:57.970757 25726 tablet_copy_client.cc:647] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b443e68b6212450286869124684bdfff: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:57.971279 25731 ts_tablet_manager.cc:1888] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.9
I20240502 07:59:57.971454 25731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bc5937510214c3da7731d3ad2a67842. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:57.972580 25731 tablet_copy_client.cc:783] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:57.972730 25731 tablet_copy_client.cc:647] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:57.973749 25726 tablet_copy_client.cc:523] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b443e68b6212450286869124684bdfff: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:57.974138 25731 tablet_copy_client.cc:523] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:57.975329 25726 tablet_bootstrap.cc:492] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b443e68b6212450286869124684bdfff: Bootstrap starting.
I20240502 07:59:57.975494 25731 tablet_bootstrap.cc:492] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191: Bootstrap starting.
I20240502 07:59:57.978220 25700 tablet_bootstrap.cc:492] T 574fdd7a3c114ac9928ac12d1a7a4186 P ff9f2ccccdfa48a1afbc0e371f41c662: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:57.978444 25700 tablet_bootstrap.cc:492] T 574fdd7a3c114ac9928ac12d1a7a4186 P ff9f2ccccdfa48a1afbc0e371f41c662: Bootstrap complete.
I20240502 07:59:57.978601 25700 ts_tablet_manager.cc:1367] T 574fdd7a3c114ac9928ac12d1a7a4186 P ff9f2ccccdfa48a1afbc0e371f41c662: Time spent bootstrapping tablet: real 0.023s user 0.021s sys 0.000s
I20240502 07:59:57.978989 25700 raft_consensus.cc:348] T 574fdd7a3c114ac9928ac12d1a7a4186 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } }
I20240502 07:59:57.979379 25700 raft_consensus.cc:729] T 574fdd7a3c114ac9928ac12d1a7a4186 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ff9f2ccccdfa48a1afbc0e371f41c662, State: Initialized, Role: LEARNER
I20240502 07:59:57.979533 25700 consensus_queue.cc:260] T 574fdd7a3c114ac9928ac12d1a7a4186 P ff9f2ccccdfa48a1afbc0e371f41c662 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } }
I20240502 07:59:57.979941 25700 ts_tablet_manager.cc:1398] T 574fdd7a3c114ac9928ac12d1a7a4186 P ff9f2ccccdfa48a1afbc0e371f41c662: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240502 07:59:57.983881 20427 tablet_copy_service.cc:342] P 35642aa2099c40e59bc2092604078e07: Request end of tablet copy session ff9f2ccccdfa48a1afbc0e371f41c662-574fdd7a3c114ac9928ac12d1a7a4186 received from {username='slave'} at 127.3.139.74:34321
I20240502 07:59:57.983960 20427 tablet_copy_service.cc:433] P 35642aa2099c40e59bc2092604078e07: ending tablet copy session ff9f2ccccdfa48a1afbc0e371f41c662-574fdd7a3c114ac9928ac12d1a7a4186 on tablet 574fdd7a3c114ac9928ac12d1a7a4186 with peer ff9f2ccccdfa48a1afbc0e371f41c662
I20240502 07:59:57.988776 25738 ts_tablet_manager.cc:902] T 880876d792554606a7ccf21f9ec08e0f P b443e68b6212450286869124684bdfff: Initiating tablet copy from peer 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703)
I20240502 07:59:57.989631 25738 tablet_copy_client.cc:308] T 880876d792554606a7ccf21f9ec08e0f P b443e68b6212450286869124684bdfff: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.65:44703
I20240502 07:59:57.990051 20163 tablet_copy_service.cc:140] P 29a7180fe4574c09a8dce7c27500aab2: Received BeginTabletCopySession request for tablet 880876d792554606a7ccf21f9ec08e0f from peer b443e68b6212450286869124684bdfff ({username='slave'} at 127.3.139.69:43889)
I20240502 07:59:57.990146 20163 tablet_copy_service.cc:161] P 29a7180fe4574c09a8dce7c27500aab2: Beginning new tablet copy session on tablet 880876d792554606a7ccf21f9ec08e0f from peer b443e68b6212450286869124684bdfff at {username='slave'} at 127.3.139.69:43889: session id = b443e68b6212450286869124684bdfff-880876d792554606a7ccf21f9ec08e0f
I20240502 07:59:57.991029 20163 tablet_copy_source_session.cc:205] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:57.991822 25738 ts_tablet_manager.cc:1875] T 880876d792554606a7ccf21f9ec08e0f P b443e68b6212450286869124684bdfff: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240502 07:59:57.995004 25738 ts_tablet_manager.cc:1888] T 880876d792554606a7ccf21f9ec08e0f P b443e68b6212450286869124684bdfff: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.6
I20240502 07:59:57.995193 25738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 880876d792554606a7ccf21f9ec08e0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:57.996279 25738 tablet_copy_client.cc:783] T 880876d792554606a7ccf21f9ec08e0f P b443e68b6212450286869124684bdfff: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:57.996405 25738 tablet_copy_client.cc:647] T 880876d792554606a7ccf21f9ec08e0f P b443e68b6212450286869124684bdfff: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:57.997438 25738 tablet_copy_client.cc:523] T 880876d792554606a7ccf21f9ec08e0f P b443e68b6212450286869124684bdfff: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:57.998492 25738 tablet_bootstrap.cc:492] T 880876d792554606a7ccf21f9ec08e0f P b443e68b6212450286869124684bdfff: Bootstrap starting.
I20240502 07:59:58.005038 25744 ts_tablet_manager.cc:902] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2: Initiating tablet copy from peer b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331)
I20240502 07:59:58.005879 25744 tablet_copy_client.cc:308] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.66:45331
I20240502 07:59:58.006500 20295 tablet_copy_service.cc:140] P b7604c0eb3bf4a7cb260b564a94a1191: Received BeginTabletCopySession request for tablet 0e2908de10684c84af542f16ff8727cb from peer 29a7180fe4574c09a8dce7c27500aab2 ({username='slave'} at 127.3.139.65:60901)
I20240502 07:59:58.006600 20295 tablet_copy_service.cc:161] P b7604c0eb3bf4a7cb260b564a94a1191: Beginning new tablet copy session on tablet 0e2908de10684c84af542f16ff8727cb from peer 29a7180fe4574c09a8dce7c27500aab2 at {username='slave'} at 127.3.139.65:60901: session id = 29a7180fe4574c09a8dce7c27500aab2-0e2908de10684c84af542f16ff8727cb
I20240502 07:59:58.006888 25731 tablet_bootstrap.cc:492] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:58.007192 25731 tablet_bootstrap.cc:492] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191: Bootstrap complete.
I20240502 07:59:58.007370 25731 ts_tablet_manager.cc:1367] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191: Time spent bootstrapping tablet: real 0.032s user 0.027s sys 0.004s
I20240502 07:59:58.007556 20295 tablet_copy_source_session.cc:205] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:58.007843 25731 raft_consensus.cc:348] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: NON_VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { promote: true } }
I20240502 07:59:58.007992 25731 raft_consensus.cc:729] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b7604c0eb3bf4a7cb260b564a94a1191, State: Initialized, Role: LEARNER
I20240502 07:59:58.008097 25731 consensus_queue.cc:260] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 1.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: NON_VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { promote: true } }
I20240502 07:59:58.008410 25731 ts_tablet_manager.cc:1398] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:58.009339 20691 tablet_copy_service.cc:342] P b443e68b6212450286869124684bdfff: Request end of tablet copy session b7604c0eb3bf4a7cb260b564a94a1191-1bc5937510214c3da7731d3ad2a67842 received from {username='slave'} at 127.3.139.66:48499
I20240502 07:59:58.009414 20691 tablet_copy_service.cc:433] P b443e68b6212450286869124684bdfff: ending tablet copy session b7604c0eb3bf4a7cb260b564a94a1191-1bc5937510214c3da7731d3ad2a67842 on tablet 1bc5937510214c3da7731d3ad2a67842 with peer b7604c0eb3bf4a7cb260b564a94a1191
I20240502 07:59:58.010138 25744 ts_tablet_manager.cc:1875] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240502 07:59:58.010880 25726 tablet_bootstrap.cc:492] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b443e68b6212450286869124684bdfff: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:58.011193 25726 tablet_bootstrap.cc:492] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b443e68b6212450286869124684bdfff: Bootstrap complete.
I20240502 07:59:58.011372 25726 ts_tablet_manager.cc:1367] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b443e68b6212450286869124684bdfff: Time spent bootstrapping tablet: real 0.036s user 0.031s sys 0.000s
I20240502 07:59:58.011848 25726 raft_consensus.cc:348] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b443e68b6212450286869124684bdfff [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: true } }
I20240502 07:59:58.011994 25726 raft_consensus.cc:729] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b443e68b6212450286869124684bdfff [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b443e68b6212450286869124684bdfff, State: Initialized, Role: LEARNER
I20240502 07:59:58.012099 25726 consensus_queue.cc:260] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b443e68b6212450286869124684bdfff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 1.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: true } }
I20240502 07:59:58.012460 25726 ts_tablet_manager.cc:1398] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b443e68b6212450286869124684bdfff: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:58.013176 25744 ts_tablet_manager.cc:1888] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.9
I20240502 07:59:58.013239 20559 tablet_copy_service.cc:342] P e951f009258e473cb57cf8773c7975d4: Request end of tablet copy session b443e68b6212450286869124684bdfff-dbf4b90cac9e4ff3b2936a1fa2ca8561 received from {username='slave'} at 127.3.139.69:33433
I20240502 07:59:58.013315 20559 tablet_copy_service.cc:433] P e951f009258e473cb57cf8773c7975d4: ending tablet copy session b443e68b6212450286869124684bdfff-dbf4b90cac9e4ff3b2936a1fa2ca8561 on tablet dbf4b90cac9e4ff3b2936a1fa2ca8561 with peer b443e68b6212450286869124684bdfff
I20240502 07:59:58.013346 25744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e2908de10684c84af542f16ff8727cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:58.015285 25744 tablet_copy_client.cc:783] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:58.015457 25744 tablet_copy_client.cc:647] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:58.017252 25744 tablet_copy_client.cc:523] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:58.018532 25744 tablet_bootstrap.cc:492] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2: Bootstrap starting.
I20240502 07:59:58.022383 25749 ts_tablet_manager.cc:902] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07: Initiating tablet copy from peer b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577)
I20240502 07:59:58.023202 25749 tablet_copy_client.cc:308] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.71:39577
I20240502 07:59:58.023640 25738 tablet_bootstrap.cc:492] T 880876d792554606a7ccf21f9ec08e0f P b443e68b6212450286869124684bdfff: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:58.023900 25738 tablet_bootstrap.cc:492] T 880876d792554606a7ccf21f9ec08e0f P b443e68b6212450286869124684bdfff: Bootstrap complete.
I20240502 07:59:58.024086 25738 ts_tablet_manager.cc:1367] T 880876d792554606a7ccf21f9ec08e0f P b443e68b6212450286869124684bdfff: Time spent bootstrapping tablet: real 0.026s user 0.022s sys 0.001s
I20240502 07:59:58.024379 20955 tablet_copy_service.cc:140] P b5f471f8b5944986a7d878cf6ac550b8: Received BeginTabletCopySession request for tablet f588a3ec49a24b1a9a9e19cdf8c30ca5 from peer 35642aa2099c40e59bc2092604078e07 ({username='slave'} at 127.3.139.67:56721)
I20240502 07:59:58.024483 20955 tablet_copy_service.cc:161] P b5f471f8b5944986a7d878cf6ac550b8: Beginning new tablet copy session on tablet f588a3ec49a24b1a9a9e19cdf8c30ca5 from peer 35642aa2099c40e59bc2092604078e07 at {username='slave'} at 127.3.139.67:56721: session id = 35642aa2099c40e59bc2092604078e07-f588a3ec49a24b1a9a9e19cdf8c30ca5
I20240502 07:59:58.024539 25738 raft_consensus.cc:348] T 880876d792554606a7ccf21f9ec08e0f P b443e68b6212450286869124684bdfff [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: true } }
I20240502 07:59:58.024652 25738 raft_consensus.cc:729] T 880876d792554606a7ccf21f9ec08e0f P b443e68b6212450286869124684bdfff [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b443e68b6212450286869124684bdfff, State: Initialized, Role: LEARNER
I20240502 07:59:58.024724 25738 consensus_queue.cc:260] T 880876d792554606a7ccf21f9ec08e0f P b443e68b6212450286869124684bdfff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: true } }
I20240502 07:59:58.024986 25738 ts_tablet_manager.cc:1398] T 880876d792554606a7ccf21f9ec08e0f P b443e68b6212450286869124684bdfff: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:58.025347 20955 tablet_copy_source_session.cc:205] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:58.025417 20163 tablet_copy_service.cc:342] P 29a7180fe4574c09a8dce7c27500aab2: Request end of tablet copy session b443e68b6212450286869124684bdfff-880876d792554606a7ccf21f9ec08e0f received from {username='slave'} at 127.3.139.69:43889
I20240502 07:59:58.025482 20163 tablet_copy_service.cc:433] P 29a7180fe4574c09a8dce7c27500aab2: ending tablet copy session b443e68b6212450286869124684bdfff-880876d792554606a7ccf21f9ec08e0f on tablet 880876d792554606a7ccf21f9ec08e0f with peer b443e68b6212450286869124684bdfff
I20240502 07:59:58.026562 25749 ts_tablet_manager.cc:1875] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240502 07:59:58.028841 25749 ts_tablet_manager.cc:1888] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.9
I20240502 07:59:58.029014 25749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f588a3ec49a24b1a9a9e19cdf8c30ca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:58.030146 25749 tablet_copy_client.cc:783] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:58.030288 25749 tablet_copy_client.cc:647] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:58.032322 25749 tablet_copy_client.cc:523] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:58.033628 25749 tablet_bootstrap.cc:492] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07: Bootstrap starting.
I20240502 07:59:58.046923 25744 tablet_bootstrap.cc:492] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:58.047433 25744 tablet_bootstrap.cc:492] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2: Bootstrap complete.
I20240502 07:59:58.047603 25744 ts_tablet_manager.cc:1367] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2: Time spent bootstrapping tablet: real 0.029s user 0.025s sys 0.000s
I20240502 07:59:58.048004 25744 raft_consensus.cc:348] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: NON_VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { promote: true } }
I20240502 07:59:58.048125 25744 raft_consensus.cc:729] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 29a7180fe4574c09a8dce7c27500aab2, State: Initialized, Role: LEARNER
I20240502 07:59:58.048219 25744 consensus_queue.cc:260] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 1.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: NON_VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { promote: true } }
I20240502 07:59:58.048472 25744 ts_tablet_manager.cc:1398] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:58.049456 20295 tablet_copy_service.cc:342] P b7604c0eb3bf4a7cb260b564a94a1191: Request end of tablet copy session 29a7180fe4574c09a8dce7c27500aab2-0e2908de10684c84af542f16ff8727cb received from {username='slave'} at 127.3.139.65:60901
I20240502 07:59:58.049547 20295 tablet_copy_service.cc:433] P b7604c0eb3bf4a7cb260b564a94a1191: ending tablet copy session 29a7180fe4574c09a8dce7c27500aab2-0e2908de10684c84af542f16ff8727cb on tablet 0e2908de10684c84af542f16ff8727cb with peer 29a7180fe4574c09a8dce7c27500aab2
I20240502 07:59:58.057080 25749 tablet_bootstrap.cc:492] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:58.057379 25749 tablet_bootstrap.cc:492] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07: Bootstrap complete.
I20240502 07:59:58.057580 25749 ts_tablet_manager.cc:1367] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07: Time spent bootstrapping tablet: real 0.024s user 0.023s sys 0.000s
I20240502 07:59:58.058045 25749 raft_consensus.cc:348] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: NON_VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: true } }
I20240502 07:59:58.058197 25749 raft_consensus.cc:729] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 35642aa2099c40e59bc2092604078e07, State: Initialized, Role: LEARNER
I20240502 07:59:58.058297 25749 consensus_queue.cc:260] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 1.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: NON_VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: true } }
I20240502 07:59:58.058624 25749 ts_tablet_manager.cc:1398] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:58.059250 20955 tablet_copy_service.cc:342] P b5f471f8b5944986a7d878cf6ac550b8: Request end of tablet copy session 35642aa2099c40e59bc2092604078e07-f588a3ec49a24b1a9a9e19cdf8c30ca5 received from {username='slave'} at 127.3.139.67:56721
I20240502 07:59:58.059330 20955 tablet_copy_service.cc:433] P b5f471f8b5944986a7d878cf6ac550b8: ending tablet copy session 35642aa2099c40e59bc2092604078e07-f588a3ec49a24b1a9a9e19cdf8c30ca5 on tablet f588a3ec49a24b1a9a9e19cdf8c30ca5 with peer 35642aa2099c40e59bc2092604078e07
I20240502 07:59:58.077383 25781 ts_tablet_manager.cc:902] T 6a77bda800d44c4bbe1b082a5ea05b55 P e951f009258e473cb57cf8773c7975d4: Initiating tablet copy from peer c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085)
I20240502 07:59:58.078250 25781 tablet_copy_client.cc:308] T 6a77bda800d44c4bbe1b082a5ea05b55 P e951f009258e473cb57cf8773c7975d4: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.70:36085
I20240502 07:59:58.078991 20823 tablet_copy_service.cc:140] P c15e23a34059456788f9775031e27ec0: Received BeginTabletCopySession request for tablet 6a77bda800d44c4bbe1b082a5ea05b55 from peer e951f009258e473cb57cf8773c7975d4 ({username='slave'} at 127.3.139.68:49059)
I20240502 07:59:58.079118 20823 tablet_copy_service.cc:161] P c15e23a34059456788f9775031e27ec0: Beginning new tablet copy session on tablet 6a77bda800d44c4bbe1b082a5ea05b55 from peer e951f009258e473cb57cf8773c7975d4 at {username='slave'} at 127.3.139.68:49059: session id = e951f009258e473cb57cf8773c7975d4-6a77bda800d44c4bbe1b082a5ea05b55
I20240502 07:59:58.080060 20823 tablet_copy_source_session.cc:205] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:58.081053 25781 ts_tablet_manager.cc:1875] T 6a77bda800d44c4bbe1b082a5ea05b55 P e951f009258e473cb57cf8773c7975d4: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240502 07:59:58.083482 25781 ts_tablet_manager.cc:1888] T 6a77bda800d44c4bbe1b082a5ea05b55 P e951f009258e473cb57cf8773c7975d4: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.6
I20240502 07:59:58.083628 25781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a77bda800d44c4bbe1b082a5ea05b55. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:58.084535 25781 tablet_copy_client.cc:783] T 6a77bda800d44c4bbe1b082a5ea05b55 P e951f009258e473cb57cf8773c7975d4: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:58.084690 25781 tablet_copy_client.cc:647] T 6a77bda800d44c4bbe1b082a5ea05b55 P e951f009258e473cb57cf8773c7975d4: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:58.085945 25781 tablet_copy_client.cc:523] T 6a77bda800d44c4bbe1b082a5ea05b55 P e951f009258e473cb57cf8773c7975d4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:58.087430 25781 tablet_bootstrap.cc:492] T 6a77bda800d44c4bbe1b082a5ea05b55 P e951f009258e473cb57cf8773c7975d4: Bootstrap starting.
I20240502 07:59:58.087493 25738 ts_tablet_manager.cc:902] T fb0f04b72dc046c0afee40b895e17a1d P b443e68b6212450286869124684bdfff: Initiating tablet copy from peer e951f009258e473cb57cf8773c7975d4 (127.3.139.68:40315)
I20240502 07:59:58.087965 25738 tablet_copy_client.cc:308] T fb0f04b72dc046c0afee40b895e17a1d P b443e68b6212450286869124684bdfff: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.68:40315
I20240502 07:59:58.088405 20559 tablet_copy_service.cc:140] P e951f009258e473cb57cf8773c7975d4: Received BeginTabletCopySession request for tablet fb0f04b72dc046c0afee40b895e17a1d from peer b443e68b6212450286869124684bdfff ({username='slave'} at 127.3.139.69:33433)
I20240502 07:59:58.088500 20559 tablet_copy_service.cc:161] P e951f009258e473cb57cf8773c7975d4: Beginning new tablet copy session on tablet fb0f04b72dc046c0afee40b895e17a1d from peer b443e68b6212450286869124684bdfff at {username='slave'} at 127.3.139.69:33433: session id = b443e68b6212450286869124684bdfff-fb0f04b72dc046c0afee40b895e17a1d
I20240502 07:59:58.089315 20559 tablet_copy_source_session.cc:205] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:58.089926 25738 ts_tablet_manager.cc:1875] T fb0f04b72dc046c0afee40b895e17a1d P b443e68b6212450286869124684bdfff: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240502 07:59:58.091780 25738 ts_tablet_manager.cc:1888] T fb0f04b72dc046c0afee40b895e17a1d P b443e68b6212450286869124684bdfff: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.6
I20240502 07:59:58.091954 25738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb0f04b72dc046c0afee40b895e17a1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:58.092891 25738 tablet_copy_client.cc:783] T fb0f04b72dc046c0afee40b895e17a1d P b443e68b6212450286869124684bdfff: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:58.093024 25738 tablet_copy_client.cc:647] T fb0f04b72dc046c0afee40b895e17a1d P b443e68b6212450286869124684bdfff: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:58.094019 25738 tablet_copy_client.cc:523] T fb0f04b72dc046c0afee40b895e17a1d P b443e68b6212450286869124684bdfff: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:58.094911 25738 tablet_bootstrap.cc:492] T fb0f04b72dc046c0afee40b895e17a1d P b443e68b6212450286869124684bdfff: Bootstrap starting.
I20240502 07:59:58.110733 25781 tablet_bootstrap.cc:492] T 6a77bda800d44c4bbe1b082a5ea05b55 P e951f009258e473cb57cf8773c7975d4: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:58.111116 25781 tablet_bootstrap.cc:492] T 6a77bda800d44c4bbe1b082a5ea05b55 P e951f009258e473cb57cf8773c7975d4: Bootstrap complete.
I20240502 07:59:58.111321 25781 ts_tablet_manager.cc:1367] T 6a77bda800d44c4bbe1b082a5ea05b55 P e951f009258e473cb57cf8773c7975d4: Time spent bootstrapping tablet: real 0.024s user 0.023s sys 0.000s
I20240502 07:59:58.111809 25781 raft_consensus.cc:348] T 6a77bda800d44c4bbe1b082a5ea05b55 P e951f009258e473cb57cf8773c7975d4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: NON_VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { promote: true } }
I20240502 07:59:58.111958 25781 raft_consensus.cc:729] T 6a77bda800d44c4bbe1b082a5ea05b55 P e951f009258e473cb57cf8773c7975d4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e951f009258e473cb57cf8773c7975d4, State: Initialized, Role: LEARNER
I20240502 07:59:58.112072 25781 consensus_queue.cc:260] T 6a77bda800d44c4bbe1b082a5ea05b55 P e951f009258e473cb57cf8773c7975d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: NON_VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { promote: true } }
I20240502 07:59:58.112454 25781 ts_tablet_manager.cc:1398] T 6a77bda800d44c4bbe1b082a5ea05b55 P e951f009258e473cb57cf8773c7975d4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:58.115613 20823 tablet_copy_service.cc:342] P c15e23a34059456788f9775031e27ec0: Request end of tablet copy session e951f009258e473cb57cf8773c7975d4-6a77bda800d44c4bbe1b082a5ea05b55 received from {username='slave'} at 127.3.139.68:49059
I20240502 07:59:58.115710 20823 tablet_copy_service.cc:433] P c15e23a34059456788f9775031e27ec0: ending tablet copy session e951f009258e473cb57cf8773c7975d4-6a77bda800d44c4bbe1b082a5ea05b55 on tablet 6a77bda800d44c4bbe1b082a5ea05b55 with peer e951f009258e473cb57cf8773c7975d4
I20240502 07:59:58.119246 25797 ts_tablet_manager.cc:902] T e61fe3cb61c240f9b2c898aff1951144 P f014c4ad4c1342219a1004181ac33ed8: Initiating tablet copy from peer 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587)
I20240502 07:59:58.119796 25797 tablet_copy_client.cc:308] T e61fe3cb61c240f9b2c898aff1951144 P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.67:45587
I20240502 07:59:58.120265 20427 tablet_copy_service.cc:140] P 35642aa2099c40e59bc2092604078e07: Received BeginTabletCopySession request for tablet e61fe3cb61c240f9b2c898aff1951144 from peer f014c4ad4c1342219a1004181ac33ed8 ({username='slave'} at 127.3.139.72:45177)
I20240502 07:59:58.120393 20427 tablet_copy_service.cc:161] P 35642aa2099c40e59bc2092604078e07: Beginning new tablet copy session on tablet e61fe3cb61c240f9b2c898aff1951144 from peer f014c4ad4c1342219a1004181ac33ed8 at {username='slave'} at 127.3.139.72:45177: session id = f014c4ad4c1342219a1004181ac33ed8-e61fe3cb61c240f9b2c898aff1951144
I20240502 07:59:58.121086 20427 tablet_copy_source_session.cc:205] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:58.121945 25797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e61fe3cb61c240f9b2c898aff1951144. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:58.124187 25797 tablet_copy_client.cc:783] T e61fe3cb61c240f9b2c898aff1951144 P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:58.124321 25797 tablet_copy_client.cc:647] T e61fe3cb61c240f9b2c898aff1951144 P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:58.125134 25738 tablet_bootstrap.cc:492] T fb0f04b72dc046c0afee40b895e17a1d P b443e68b6212450286869124684bdfff: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:58.125376 25738 tablet_bootstrap.cc:492] T fb0f04b72dc046c0afee40b895e17a1d P b443e68b6212450286869124684bdfff: Bootstrap complete.
I20240502 07:59:58.125551 25738 ts_tablet_manager.cc:1367] T fb0f04b72dc046c0afee40b895e17a1d P b443e68b6212450286869124684bdfff: Time spent bootstrapping tablet: real 0.031s user 0.022s sys 0.000s
I20240502 07:59:58.125964 25738 raft_consensus.cc:348] T fb0f04b72dc046c0afee40b895e17a1d P b443e68b6212450286869124684bdfff [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: true } }
I20240502 07:59:58.126093 25738 raft_consensus.cc:729] T fb0f04b72dc046c0afee40b895e17a1d P b443e68b6212450286869124684bdfff [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b443e68b6212450286869124684bdfff, State: Initialized, Role: LEARNER
I20240502 07:59:58.126188 25738 consensus_queue.cc:260] T fb0f04b72dc046c0afee40b895e17a1d P b443e68b6212450286869124684bdfff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: NON_VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: true } }
I20240502 07:59:58.126478 25738 ts_tablet_manager.cc:1398] T fb0f04b72dc046c0afee40b895e17a1d P b443e68b6212450286869124684bdfff: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:58.127841 20559 tablet_copy_service.cc:342] P e951f009258e473cb57cf8773c7975d4: Request end of tablet copy session b443e68b6212450286869124684bdfff-fb0f04b72dc046c0afee40b895e17a1d received from {username='slave'} at 127.3.139.69:33433
I20240502 07:59:58.127904 20559 tablet_copy_service.cc:433] P e951f009258e473cb57cf8773c7975d4: ending tablet copy session b443e68b6212450286869124684bdfff-fb0f04b72dc046c0afee40b895e17a1d on tablet fb0f04b72dc046c0afee40b895e17a1d with peer b443e68b6212450286869124684bdfff
I20240502 07:59:58.129585 25797 tablet_copy_client.cc:523] T e61fe3cb61c240f9b2c898aff1951144 P f014c4ad4c1342219a1004181ac33ed8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:58.131611 25797 tablet_bootstrap.cc:492] T e61fe3cb61c240f9b2c898aff1951144 P f014c4ad4c1342219a1004181ac33ed8: Bootstrap starting.
I20240502 07:59:58.148214 25809 ts_tablet_manager.cc:902] T bbae35bae8c047a6982e128ffb77ca86 P c15e23a34059456788f9775031e27ec0: Initiating tablet copy from peer b443e68b6212450286869124684bdfff (127.3.139.69:37923)
I20240502 07:59:58.148993 25809 tablet_copy_client.cc:308] T bbae35bae8c047a6982e128ffb77ca86 P c15e23a34059456788f9775031e27ec0: tablet copy: Beginning tablet copy session from remote peer at address 127.3.139.69:37923
I20240502 07:59:58.149624 20691 tablet_copy_service.cc:140] P b443e68b6212450286869124684bdfff: Received BeginTabletCopySession request for tablet bbae35bae8c047a6982e128ffb77ca86 from peer c15e23a34059456788f9775031e27ec0 ({username='slave'} at 127.3.139.70:34075)
I20240502 07:59:58.149729 20691 tablet_copy_service.cc:161] P b443e68b6212450286869124684bdfff: Beginning new tablet copy session on tablet bbae35bae8c047a6982e128ffb77ca86 from peer c15e23a34059456788f9775031e27ec0 at {username='slave'} at 127.3.139.70:34075: session id = c15e23a34059456788f9775031e27ec0-bbae35bae8c047a6982e128ffb77ca86
I20240502 07:59:58.150590 20691 tablet_copy_source_session.cc:205] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:58.151654 25809 ts_tablet_manager.cc:1875] T bbae35bae8c047a6982e128ffb77ca86 P c15e23a34059456788f9775031e27ec0: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240502 07:59:58.154805 25809 ts_tablet_manager.cc:1888] T bbae35bae8c047a6982e128ffb77ca86 P c15e23a34059456788f9775031e27ec0: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.6
I20240502 07:59:58.155117 25809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbae35bae8c047a6982e128ffb77ca86. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:58.163705 25797 tablet_bootstrap.cc:492] T e61fe3cb61c240f9b2c898aff1951144 P f014c4ad4c1342219a1004181ac33ed8: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:58.163997 25797 tablet_bootstrap.cc:492] T e61fe3cb61c240f9b2c898aff1951144 P f014c4ad4c1342219a1004181ac33ed8: Bootstrap complete.
I20240502 07:59:58.164165 25797 ts_tablet_manager.cc:1367] T e61fe3cb61c240f9b2c898aff1951144 P f014c4ad4c1342219a1004181ac33ed8: Time spent bootstrapping tablet: real 0.033s user 0.025s sys 0.001s
I20240502 07:59:58.164606 25797 raft_consensus.cc:348] T e61fe3cb61c240f9b2c898aff1951144 P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } }
I20240502 07:59:58.164760 25797 raft_consensus.cc:729] T e61fe3cb61c240f9b2c898aff1951144 P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f014c4ad4c1342219a1004181ac33ed8, State: Initialized, Role: LEARNER
I20240502 07:59:58.164866 25797 consensus_queue.cc:260] T e61fe3cb61c240f9b2c898aff1951144 P f014c4ad4c1342219a1004181ac33ed8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 1.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } }
I20240502 07:59:58.165218 25797 ts_tablet_manager.cc:1398] T e61fe3cb61c240f9b2c898aff1951144 P f014c4ad4c1342219a1004181ac33ed8: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 07:59:58.165380 25809 tablet_copy_client.cc:783] T bbae35bae8c047a6982e128ffb77ca86 P c15e23a34059456788f9775031e27ec0: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:58.165589 25809 tablet_copy_client.cc:647] T bbae35bae8c047a6982e128ffb77ca86 P c15e23a34059456788f9775031e27ec0: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:58.166106 20427 tablet_copy_service.cc:342] P 35642aa2099c40e59bc2092604078e07: Request end of tablet copy session f014c4ad4c1342219a1004181ac33ed8-e61fe3cb61c240f9b2c898aff1951144 received from {username='slave'} at 127.3.139.72:45177
I20240502 07:59:58.166184 20427 tablet_copy_service.cc:433] P 35642aa2099c40e59bc2092604078e07: ending tablet copy session f014c4ad4c1342219a1004181ac33ed8-e61fe3cb61c240f9b2c898aff1951144 on tablet e61fe3cb61c240f9b2c898aff1951144 with peer f014c4ad4c1342219a1004181ac33ed8
I20240502 07:59:58.169293 25809 tablet_copy_client.cc:523] T bbae35bae8c047a6982e128ffb77ca86 P c15e23a34059456788f9775031e27ec0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:58.172775 25809 tablet_bootstrap.cc:492] T bbae35bae8c047a6982e128ffb77ca86 P c15e23a34059456788f9775031e27ec0: Bootstrap starting.
I20240502 07:59:58.213150 25809 tablet_bootstrap.cc:492] T bbae35bae8c047a6982e128ffb77ca86 P c15e23a34059456788f9775031e27ec0: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:58.213500 25809 tablet_bootstrap.cc:492] T bbae35bae8c047a6982e128ffb77ca86 P c15e23a34059456788f9775031e27ec0: Bootstrap complete.
I20240502 07:59:58.213701 25809 ts_tablet_manager.cc:1367] T bbae35bae8c047a6982e128ffb77ca86 P c15e23a34059456788f9775031e27ec0: Time spent bootstrapping tablet: real 0.041s user 0.020s sys 0.004s
I20240502 07:59:58.214296 25809 raft_consensus.cc:348] T bbae35bae8c047a6982e128ffb77ca86 P c15e23a34059456788f9775031e27ec0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { promote: true } }
I20240502 07:59:58.214457 25809 raft_consensus.cc:729] T bbae35bae8c047a6982e128ffb77ca86 P c15e23a34059456788f9775031e27ec0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c15e23a34059456788f9775031e27ec0, State: Initialized, Role: LEARNER
I20240502 07:59:58.214566 25809 consensus_queue.cc:260] T bbae35bae8c047a6982e128ffb77ca86 P c15e23a34059456788f9775031e27ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 1.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: NON_VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { promote: true } }
I20240502 07:59:58.214965 25809 ts_tablet_manager.cc:1398] T bbae35bae8c047a6982e128ffb77ca86 P c15e23a34059456788f9775031e27ec0: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 07:59:58.215732 20691 tablet_copy_service.cc:342] P b443e68b6212450286869124684bdfff: Request end of tablet copy session c15e23a34059456788f9775031e27ec0-bbae35bae8c047a6982e128ffb77ca86 received from {username='slave'} at 127.3.139.70:34075
I20240502 07:59:58.215821 20691 tablet_copy_service.cc:433] P b443e68b6212450286869124684bdfff: ending tablet copy session c15e23a34059456788f9775031e27ec0-bbae35bae8c047a6982e128ffb77ca86 on tablet bbae35bae8c047a6982e128ffb77ca86 with peer c15e23a34059456788f9775031e27ec0
I20240502 07:59:58.331296 22593 raft_consensus.cc:1203] T a973de06330c487ba54a8a36b1e31b54 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240502 07:59:58.344835 20671 raft_consensus.cc:1203] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b443e68b6212450286869124684bdfff [term 1 LEARNER]: Deduplicated request from leader. Original: 1.10->[1.11-1.11] Dedup: 1.11->[]
I20240502 07:59:58.465818 20140 raft_consensus.cc:1203] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.10->[1.11-1.11] Dedup: 1.11->[]
I20240502 07:59:58.500320 20273 raft_consensus.cc:1203] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.10->[1.11-1.11] Dedup: 1.11->[]
I20240502 07:59:58.509933 20671 raft_consensus.cc:1203] T 880876d792554606a7ccf21f9ec08e0f P b443e68b6212450286869124684bdfff [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240502 07:59:58.526851 22593 raft_consensus.cc:1203] T 574fdd7a3c114ac9928ac12d1a7a4186 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240502 07:59:58.551816 20537 raft_consensus.cc:1203] T 6a77bda800d44c4bbe1b082a5ea05b55 P e951f009258e473cb57cf8773c7975d4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240502 07:59:58.569330 20406 raft_consensus.cc:1203] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.10->[1.11-1.11] Dedup: 1.11->[]
I20240502 07:59:58.569978 22741 raft_consensus.cc:1203] T e61fe3cb61c240f9b2c898aff1951144 P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.10->[1.11-1.11] Dedup: 1.11->[]
I20240502 07:59:58.628791 20671 raft_consensus.cc:1203] T fb0f04b72dc046c0afee40b895e17a1d P b443e68b6212450286869124684bdfff [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240502 07:59:58.684207 20801 raft_consensus.cc:1203] T bbae35bae8c047a6982e128ffb77ca86 P c15e23a34059456788f9775031e27ec0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.10->[1.11-1.11] Dedup: 1.11->[]
I20240502 07:59:58.758534 25633 raft_consensus.cc:1050] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4: attempting to promote NON_VOTER b443e68b6212450286869124684bdfff to VOTER
I20240502 07:59:58.759024 25633 consensus_queue.cc:237] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } }
I20240502 07:59:58.760877 20671 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b443e68b6212450286869124684bdfff [term 1 LEARNER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 07:59:58.760893 20801 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 07:59:58.760902 20933 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 07:59:58.761161 22593 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 07:59:58.761456 22475 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 07:59:58.761600 20272 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 07:59:58.761575 25633 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:58.761811 25093 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:58.761950 25522 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:58.762107 25633 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:58.762214 25985 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:58.762465 20140 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 07:59:58.762885 25984 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:58.763517 25633 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:58.765193 25633 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [term 1 LEADER]: Committing config change with OpId 1.12: config changed from index 11 to 12, b443e68b6212450286869124684bdfff (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:58.765841 20801 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, b443e68b6212450286869124684bdfff (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:58.767261 20933 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, b443e68b6212450286869124684bdfff (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:58.767654 20272 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, b443e68b6212450286869124684bdfff (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:58.767915 19995 catalog_manager.cc:5484] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 reported cstate change: config changed from index 11 to 12, b443e68b6212450286869124684bdfff (127.3.139.69) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e951f009258e473cb57cf8773c7975d4" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 07:59:58.768805 22475 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, b443e68b6212450286869124684bdfff (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:58.770238 22593 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, b443e68b6212450286869124684bdfff (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:58.771958 20671 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, b443e68b6212450286869124684bdfff (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:58.772330 20140 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, b443e68b6212450286869124684bdfff (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { replace: true } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:58.776959 20537 consensus_queue.cc:237] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } }
I20240502 07:59:58.778405 22593 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 07:59:58.778467 20272 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 07:59:58.778520 20933 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 07:59:58.778859 22475 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 07:59:58.779122 25093 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 07:59:58.779279 25986 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 07:59:58.779579 20671 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 07:59:58.779641 25633 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 07:59:58.779878 20140 raft_consensus.cc:1261] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 07:59:58.779917 25986 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 07:59:58.780098 25522 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 07:59:58.780560 25522 consensus_queue.cc:1035] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 07:59:58.782299 25633 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 [term 1 LEADER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:58.782580 20933 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:58.783676 20272 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:58.784339 22475 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:58.784626 20671 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:58.784682 20143 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:58.784925 19986 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet dbf4b90cac9e4ff3b2936a1fa2ca8561 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:58.787187 22593 raft_consensus.cc:2932] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:58.786902 19993 catalog_manager.cc:5484] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P bfb1b3d596ef474faa229a835329cf20 reported cstate change: config changed from index 12 to 13, VOTER c15e23a34059456788f9775031e27ec0 (127.3.139.70) evicted. New cstate: current_term: 1 leader_uuid: "e951f009258e473cb57cf8773c7975d4" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:58.789886 20781 tablet_service.cc:1514] Processing DeleteTablet for tablet dbf4b90cac9e4ff3b2936a1fa2ca8561 with delete_type TABLET_DATA_TOMBSTONED (TS c15e23a34059456788f9775031e27ec0 not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:45388
I20240502 07:59:58.790277 25999 tablet_replica.cc:330] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P c15e23a34059456788f9775031e27ec0: stopping tablet replica
I20240502 07:59:58.790589 25999 raft_consensus.cc:2229] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:58.790828 25999 raft_consensus.cc:2258] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:58.791549 25999 ts_tablet_manager.cc:1875] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P c15e23a34059456788f9775031e27ec0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:58.793745 25999 ts_tablet_manager.cc:1888] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P c15e23a34059456788f9775031e27ec0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.12
I20240502 07:59:58.793821 25999 log.cc:1196] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P c15e23a34059456788f9775031e27ec0: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-5/wal/wals/dbf4b90cac9e4ff3b2936a1fa2ca8561
I20240502 07:59:58.794363 19986 catalog_manager.cc:4830] TS c15e23a34059456788f9775031e27ec0 (127.3.139.70:36085): tablet dbf4b90cac9e4ff3b2936a1fa2ca8561 (table rebalance_test_table_2 [id=30a30b02a2404507a95fc3b5bb77bdd1]) successfully deleted
I20240502 07:59:58.842623 25633 raft_consensus.cc:1050] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4: attempting to promote NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 to VOTER
I20240502 07:59:58.843192 25633 consensus_queue.cc:237] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } }
I20240502 07:59:58.845120 22593 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:58.845132 20272 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:58.845374 20801 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:58.845530 22741 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:58.845748 25522 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:58.845918 20143 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:58.845937 25633 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:58.846092 25983 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:58.846168 20406 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:58.846285 25985 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:58.846582 25093 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:58.847488 25986 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:58.848276 20933 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:58.848726 25522 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:58.849957 25093 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:58.851855 20272 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:58.852226 20933 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:58.853319 22609 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:58.853543 22741 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:58.854525 20406 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:58.855114 19992 catalog_manager.cc:5484] T a973de06330c487ba54a8a36b1e31b54 P b7604c0eb3bf4a7cb260b564a94a1191 reported cstate change: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e951f009258e473cb57cf8773c7975d4" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:58.855208 20801 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:58.856731 20143 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:58.863190 20537 consensus_queue.cc:237] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } }
I20240502 07:59:58.864881 20801 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:58.865192 25983 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:58.865520 20933 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:58.866021 20406 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:58.866091 22609 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:58.866372 25985 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:58.866686 25985 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:58.866853 20273 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:58.866986 25984 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:58.867295 25985 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:58.868880 20143 raft_consensus.cc:1261] T a973de06330c487ba54a8a36b1e31b54 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:58.869354 25633 consensus_queue.cc:1035] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:58.870744 25984 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:58.871486 20272 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:58.871779 20143 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:58.874135 20933 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:58.874190 20407 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:58.874431 19999 catalog_manager.cc:5484] T a973de06330c487ba54a8a36b1e31b54 P 29a7180fe4574c09a8dce7c27500aab2 reported cstate change: config changed from index 9 to 10, VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) evicted. New cstate: current_term: 1 leader_uuid: "e951f009258e473cb57cf8773c7975d4" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:58.877542 20801 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:58.879285 19986 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet a973de06330c487ba54a8a36b1e31b54 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:58.879855 22609 raft_consensus.cc:2932] T a973de06330c487ba54a8a36b1e31b54 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:58.885161 22721 tablet_service.cc:1514] Processing DeleteTablet for tablet a973de06330c487ba54a8a36b1e31b54 with delete_type TABLET_DATA_TOMBSTONED (TS f014c4ad4c1342219a1004181ac33ed8 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:54852
I20240502 07:59:58.885710 26015 tablet_replica.cc:330] T a973de06330c487ba54a8a36b1e31b54 P f014c4ad4c1342219a1004181ac33ed8: stopping tablet replica
I20240502 07:59:58.886066 26015 raft_consensus.cc:2229] T a973de06330c487ba54a8a36b1e31b54 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:58.886484 26015 raft_consensus.cc:2258] T a973de06330c487ba54a8a36b1e31b54 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:58.887506 26015 ts_tablet_manager.cc:1875] T a973de06330c487ba54a8a36b1e31b54 P f014c4ad4c1342219a1004181ac33ed8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:58.889428 26015 ts_tablet_manager.cc:1888] T a973de06330c487ba54a8a36b1e31b54 P f014c4ad4c1342219a1004181ac33ed8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240502 07:59:58.889510 26015 log.cc:1196] T a973de06330c487ba54a8a36b1e31b54 P f014c4ad4c1342219a1004181ac33ed8: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/wal/wals/a973de06330c487ba54a8a36b1e31b54
I20240502 07:59:58.889966 19987 catalog_manager.cc:4830] TS f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): tablet a973de06330c487ba54a8a36b1e31b54 (table rebalance_test_table_0 [id=89c0cb0099e94f1aa93c3eb299e446f0]) successfully deleted
I20240502 07:59:58.966415 26096 raft_consensus.cc:1050] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191: attempting to promote NON_VOTER 29a7180fe4574c09a8dce7c27500aab2 to VOTER
I20240502 07:59:58.966898 26096 consensus_queue.cc:237] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { promote: false } }
I20240502 07:59:58.968616 22741 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 07:59:58.968771 20537 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 07:59:58.968842 22475 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 07:59:58.968981 20143 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2 [term 1 LEARNER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 07:59:58.969197 26100 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:58.969322 20933 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 07:59:58.969524 20671 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 07:59:58.969662 20407 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 07:59:58.969689 26102 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:58.969975 25891 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:58.970108 26102 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:58.970219 26098 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:58.972107 26101 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:58.972297 26099 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:58.973502 26102 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 LEADER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { promote: false } } }
I20240502 07:59:58.973915 20671 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { promote: false } } }
I20240502 07:59:58.975503 20143 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { promote: false } } }
I20240502 07:59:58.976045 22741 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { promote: false } } }
I20240502 07:59:58.976923 19999 catalog_manager.cc:5484] T 0e2908de10684c84af542f16ff8727cb P b443e68b6212450286869124684bdfff reported cstate change: config changed from index 11 to 12, 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { promote: false } } }
I20240502 07:59:58.977547 20537 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { promote: false } } }
I20240502 07:59:58.978629 20933 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { promote: false } } }
I20240502 07:59:58.980896 22475 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { promote: false } } }
I20240502 07:59:58.981026 20272 consensus_queue.cc:237] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { promote: false } }
I20240502 07:59:58.982959 20933 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 07:59:58.983259 20143 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 07:59:58.983433 20671 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 07:59:58.983433 22741 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 07:59:58.983786 25891 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 07:59:58.983891 26098 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 07:59:58.984134 26098 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 07:59:58.984369 26098 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 07:59:58.984405 20537 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 07:59:58.985167 26098 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 07:59:58.985950 22475 raft_consensus.cc:1261] T 0e2908de10684c84af542f16ff8727cb P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Refusing update from remote peer b7604c0eb3bf4a7cb260b564a94a1191: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 07:59:58.986917 26096 consensus_queue.cc:1035] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 07:59:58.987661 26099 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 LEADER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { promote: false } } }
I20240502 07:59:58.989069 19987 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 0e2908de10684c84af542f16ff8727cb with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:58.989012 22475 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { promote: false } } }
I20240502 07:59:58.989567 20143 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { promote: false } } }
I20240502 07:59:58.990140 20933 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { promote: false } } }
I20240502 07:59:58.990648 20671 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { promote: false } } }
I20240502 07:59:58.991102 20537 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { promote: false } } }
I20240502 07:59:58.992002 19997 catalog_manager.cc:5484] T 0e2908de10684c84af542f16ff8727cb P bfb1b3d596ef474faa229a835329cf20 reported cstate change: config changed from index 12 to 13, VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) evicted. New cstate: current_term: 1 leader_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { promote: false } } }
I20240502 07:59:58.992169 22741 raft_consensus.cc:2932] T 0e2908de10684c84af542f16ff8727cb P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 35642aa2099c40e59bc2092604078e07 (127.3.139.67) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { promote: false } } }
I20240502 07:59:58.994489 20386 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e2908de10684c84af542f16ff8727cb with delete_type TABLET_DATA_TOMBSTONED (TS 35642aa2099c40e59bc2092604078e07 not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:43238
I20240502 07:59:58.995008 26110 tablet_replica.cc:330] T 0e2908de10684c84af542f16ff8727cb P 35642aa2099c40e59bc2092604078e07: stopping tablet replica
I20240502 07:59:58.995239 26110 raft_consensus.cc:2229] T 0e2908de10684c84af542f16ff8727cb P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:58.995389 26110 pending_rounds.cc:62] T 0e2908de10684c84af542f16ff8727cb P 35642aa2099c40e59bc2092604078e07: Trying to abort 1 pending ops.
I20240502 07:59:58.995458 26110 pending_rounds.cc:68] T 0e2908de10684c84af542f16ff8727cb P 35642aa2099c40e59bc2092604078e07: Aborting op as it isn't in flight: id { term: 1 index: 12 } timestamp: 7023152328567115776 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "0e2908de10684c84af542f16ff8727cb" old_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: NON_VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { promote: true } } } new_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { promote: false } } } }
I20240502 07:59:58.995620 26110 raft_consensus.cc:2894] T 0e2908de10684c84af542f16ff8727cb P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Aborting config change with OpId 1.12: Aborted: Op aborted
I20240502 07:59:58.995704 26110 raft_consensus.cc:2866] T 0e2908de10684c84af542f16ff8727cb P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20240502 07:59:58.995783 26110 raft_consensus.cc:2258] T 0e2908de10684c84af542f16ff8727cb P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:58.996490 26110 ts_tablet_manager.cc:1875] T 0e2908de10684c84af542f16ff8727cb P 35642aa2099c40e59bc2092604078e07: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:59.000350 26110 ts_tablet_manager.cc:1888] T 0e2908de10684c84af542f16ff8727cb P 35642aa2099c40e59bc2092604078e07: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.12
I20240502 07:59:59.000416 26110 log.cc:1196] T 0e2908de10684c84af542f16ff8727cb P 35642aa2099c40e59bc2092604078e07: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-2/wal/wals/0e2908de10684c84af542f16ff8727cb
I20240502 07:59:59.000852 19988 catalog_manager.cc:4830] TS 35642aa2099c40e59bc2092604078e07 (127.3.139.67:45587): tablet 0e2908de10684c84af542f16ff8727cb (table rebalance_test_table_4 [id=8edc4db66dcd46ee8429441e0ce671a3]) successfully deleted
I20240502 07:59:59.027304 25461 raft_consensus.cc:1050] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8: attempting to promote NON_VOTER 35642aa2099c40e59bc2092604078e07 to VOTER
I20240502 07:59:59.027807 25461 consensus_queue.cc:237] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: false } }
I20240502 07:59:59.029677 20407 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07 [term 1 LEARNER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 07:59:59.030038 22741 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 07:59:59.030102 22475 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 07:59:59.030064 26115 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:59.030238 20801 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 07:59:59.030416 26115 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:59.030750 25461 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:59.030905 26115 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:59.031385 20537 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 07:59:59.031733 20143 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 07:59:59.031744 20272 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 07:59:59.031831 25925 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:59.032280 26115 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:59.033288 26114 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:59.035254 26114 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 LEADER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 35642aa2099c40e59bc2092604078e07 (127.3.139.67) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: false } } }
I20240502 07:59:59.035671 20143 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 35642aa2099c40e59bc2092604078e07 (127.3.139.67) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: false } } }
I20240502 07:59:59.036271 20272 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 35642aa2099c40e59bc2092604078e07 (127.3.139.67) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: false } } }
I20240502 07:59:59.037400 20537 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 35642aa2099c40e59bc2092604078e07 (127.3.139.67) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: false } } }
I20240502 07:59:59.037950 20801 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 35642aa2099c40e59bc2092604078e07 (127.3.139.67) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: false } } }
I20240502 07:59:59.038040 19999 catalog_manager.cc:5484] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 reported cstate change: config changed from index 11 to 12, 35642aa2099c40e59bc2092604078e07 (127.3.139.67) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b5f471f8b5944986a7d878cf6ac550b8" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:59.038193 22741 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 35642aa2099c40e59bc2092604078e07 (127.3.139.67) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: false } } }
I20240502 07:59:59.041654 20407 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 35642aa2099c40e59bc2092604078e07 (127.3.139.67) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: false } } }
I20240502 07:59:59.046164 25913 raft_consensus.cc:1050] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2: attempting to promote NON_VOTER b443e68b6212450286869124684bdfff to VOTER
I20240502 07:59:59.046593 25913 consensus_queue.cc:237] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } }
I20240502 07:59:59.048516 20671 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P b443e68b6212450286869124684bdfff [term 1 LEARNER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:59.049078 20933 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:59.049167 20407 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:59.049190 22475 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 35642aa2099c40e59bc2092604078e07 (127.3.139.67) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: false } } }
I20240502 07:59:59.049386 20801 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:59.049541 26124 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:59.049710 20537 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:59.049722 26125 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:59.050061 25913 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:59.050312 20272 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:59.050570 25913 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:59.050877 25913 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:59.050918 22609 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:59.051193 25913 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:59.052747 26124 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:59.054698 26125 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, b443e68b6212450286869124684bdfff (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:59.054859 20934 consensus_queue.cc:237] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: false } }
I20240502 07:59:59.055446 25926 raft_consensus.cc:1050] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07: attempting to promote NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 to VOTER
I20240502 07:59:59.055958 25926 consensus_queue.cc:237] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } }
I20240502 07:59:59.056612 20800 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 07:59:59.057381 20671 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, b443e68b6212450286869124684bdfff (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:59.058897 20801 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, b443e68b6212450286869124684bdfff (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:59.059656 25483 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 07:59:59.059697 22741 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 07:59:59.059711 22475 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 07:59:59.059968 20933 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, b443e68b6212450286869124684bdfff (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:59.060405 20406 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, b443e68b6212450286869124684bdfff (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:59.061376 20272 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, b443e68b6212450286869124684bdfff (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:59.061723 20143 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:59.062981 22593 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 LEARNER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:59.063170 22741 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:59.063339 20801 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:59.063594 20272 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:59.063445 22609 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, b443e68b6212450286869124684bdfff (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:59.063766 25483 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 07:59:59.064924 19997 catalog_manager.cc:5484] T 880876d792554606a7ccf21f9ec08e0f P c15e23a34059456788f9775031e27ec0 reported cstate change: config changed from index 8 to 9, b443e68b6212450286869124684bdfff (127.3.139.69) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "29a7180fe4574c09a8dce7c27500aab2" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:59.066689 25009 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:59.066954 25480 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:59.067107 25476 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:59.066987 20537 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, b443e68b6212450286869124684bdfff (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:59.067346 20407 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 07:59:59.067379 25476 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:59.067714 26114 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 07:59:59.068025 26114 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 07:59:59.070070 20933 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:59.070060 25009 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:59.070655 20671 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:59.070937 20140 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 07:59:59.071218 25009 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:59.071122 26115 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 LEADER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: false } } }
I20240502 07:59:59.071337 25483 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 07:59:59.072059 25480 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:59.072232 20140 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: false } } }
I20240502 07:59:59.072860 19988 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet f588a3ec49a24b1a9a9e19cdf8c30ca5 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:59.073423 22475 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: false } } }
I20240502 07:59:59.073592 22740 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: false } } }
I20240502 07:59:59.074143 20537 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 07:59:59.074498 25483 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 07:59:59.076035 19998 catalog_manager.cc:5484] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P bfb1b3d596ef474faa229a835329cf20 reported cstate change: config changed from index 12 to 13, VOTER b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) evicted. New cstate: current_term: 1 leader_uuid: "b5f471f8b5944986a7d878cf6ac550b8" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: false } } }
I20240502 07:59:59.076229 20407 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: false } } }
I20240502 07:59:59.078758 20253 tablet_service.cc:1514] Processing DeleteTablet for tablet f588a3ec49a24b1a9a9e19cdf8c30ca5 with delete_type TABLET_DATA_TOMBSTONED (TS b7604c0eb3bf4a7cb260b564a94a1191 not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:59920
I20240502 07:59:59.079185 26152 tablet_replica.cc:330] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b7604c0eb3bf4a7cb260b564a94a1191: stopping tablet replica
I20240502 07:59:59.079378 26152 raft_consensus.cc:2229] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:59.079911 26152 raft_consensus.cc:2258] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:59.081241 26141 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:59.081600 26152 ts_tablet_manager.cc:1875] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b7604c0eb3bf4a7cb260b564a94a1191: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:59.082103 20800 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: false } } }
I20240502 07:59:59.082367 20537 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: false } } }
I20240502 07:59:59.083618 20143 consensus_queue.cc:237] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } }
I20240502 07:59:59.085495 20933 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:59.086167 20537 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:59.086331 22609 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:59.086601 20272 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:59.087096 26129 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:59.087754 20671 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:59.088423 20273 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:59.088349 19994 catalog_manager.cc:5484] T 574fdd7a3c114ac9928ac12d1a7a4186 P b5f471f8b5944986a7d878cf6ac550b8 reported cstate change: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "35642aa2099c40e59bc2092604078e07" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:59.089139 26129 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:59.089286 26128 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:59.090569 20407 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:59.090791 20143 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:59.091506 20800 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:59.092150 26131 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:59.092639 26128 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:59.092844 20671 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:59.093411 22609 raft_consensus.cc:1261] T 880876d792554606a7ccf21f9ec08e0f P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Refusing update from remote peer 29a7180fe4574c09a8dce7c27500aab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:59.094715 26124 consensus_queue.cc:1035] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:59.094591 26128 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:59.095429 20407 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:59.095667 20800 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:59.095741 22609 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:59.097417 20273 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:59.097810 19986 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 880876d792554606a7ccf21f9ec08e0f with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:59.098196 20671 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:59.100170 22741 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:59.101151 20800 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:59.102146 26152 ts_tablet_manager.cc:1888] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b7604c0eb3bf4a7cb260b564a94a1191: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.12
I20240502 07:59:59.102208 26152 log.cc:1196] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b7604c0eb3bf4a7cb260b564a94a1191: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-1/wal/wals/f588a3ec49a24b1a9a9e19cdf8c30ca5
I20240502 07:59:59.102674 19987 catalog_manager.cc:4830] TS b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66:45331): tablet f588a3ec49a24b1a9a9e19cdf8c30ca5 (table rebalance_test_table_2 [id=30a30b02a2404507a95fc3b5bb77bdd1]) successfully deleted
I20240502 07:59:59.103806 20537 raft_consensus.cc:2932] T 880876d792554606a7ccf21f9ec08e0f P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:59.098243 19994 catalog_manager.cc:5484] T 880876d792554606a7ccf21f9ec08e0f P ff9f2ccccdfa48a1afbc0e371f41c662 reported cstate change: config changed from index 9 to 10, VOTER b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71) evicted. New cstate: current_term: 1 leader_uuid: "29a7180fe4574c09a8dce7c27500aab2" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:59.106216 20407 consensus_queue.cc:237] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } }
I20240502 07:59:59.107942 20273 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:59.108443 22609 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:59.108449 20933 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:59.109277 25476 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:59.109704 25476 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:59.110570 25009 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:59.112193 20915 tablet_service.cc:1514] Processing DeleteTablet for tablet 880876d792554606a7ccf21f9ec08e0f with delete_type TABLET_DATA_TOMBSTONED (TS b5f471f8b5944986a7d878cf6ac550b8 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:41530
I20240502 07:59:59.112600 26153 tablet_replica.cc:330] T 880876d792554606a7ccf21f9ec08e0f P b5f471f8b5944986a7d878cf6ac550b8: stopping tablet replica
I20240502 07:59:59.112732 26153 raft_consensus.cc:2229] T 880876d792554606a7ccf21f9ec08e0f P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:59.112988 26153 raft_consensus.cc:2258] T 880876d792554606a7ccf21f9ec08e0f P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:59.114236 20671 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:59.114593 26153 ts_tablet_manager.cc:1875] T 880876d792554606a7ccf21f9ec08e0f P b5f471f8b5944986a7d878cf6ac550b8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:59.114629 25480 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:59.115482 20143 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:59.115847 25480 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:59.116319 20800 raft_consensus.cc:1261] T 574fdd7a3c114ac9928ac12d1a7a4186 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:59.116691 25009 consensus_queue.cc:1035] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:59.117187 26153 ts_tablet_manager.cc:1888] T 880876d792554606a7ccf21f9ec08e0f P b5f471f8b5944986a7d878cf6ac550b8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240502 07:59:59.117098 25009 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:59.117280 26153 log.cc:1196] T 880876d792554606a7ccf21f9ec08e0f P b5f471f8b5944986a7d878cf6ac550b8: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/wal/wals/880876d792554606a7ccf21f9ec08e0f
I20240502 07:59:59.117836 19988 catalog_manager.cc:4830] TS b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577): tablet 880876d792554606a7ccf21f9ec08e0f (table rebalance_test_table_2 [id=30a30b02a2404507a95fc3b5bb77bdd1]) successfully deleted
I20240502 07:59:59.118121 20143 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:59.118786 19988 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 574fdd7a3c114ac9928ac12d1a7a4186 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:59.119444 20273 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:59.119673 19995 catalog_manager.cc:5484] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 reported cstate change: config changed from index 9 to 10, VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) evicted. New cstate: current_term: 1 leader_uuid: "35642aa2099c40e59bc2092604078e07" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:59.119372 20671 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:59.119956 20800 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:59.123250 22721 tablet_service.cc:1514] Processing DeleteTablet for tablet 574fdd7a3c114ac9928ac12d1a7a4186 with delete_type TABLET_DATA_TOMBSTONED (TS f014c4ad4c1342219a1004181ac33ed8 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:54852
I20240502 07:59:59.123474 26015 tablet_replica.cc:330] T 574fdd7a3c114ac9928ac12d1a7a4186 P f014c4ad4c1342219a1004181ac33ed8: stopping tablet replica
I20240502 07:59:59.123605 26015 raft_consensus.cc:2229] T 574fdd7a3c114ac9928ac12d1a7a4186 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:59.123768 26015 raft_consensus.cc:2258] T 574fdd7a3c114ac9928ac12d1a7a4186 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:59.123694 22609 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:59.124718 26015 ts_tablet_manager.cc:1875] T 574fdd7a3c114ac9928ac12d1a7a4186 P f014c4ad4c1342219a1004181ac33ed8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:59.126698 25806 raft_consensus.cc:1050] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff: attempting to promote NON_VOTER b7604c0eb3bf4a7cb260b564a94a1191 to VOTER
I20240502 07:59:59.126752 26015 ts_tablet_manager.cc:1888] T 574fdd7a3c114ac9928ac12d1a7a4186 P f014c4ad4c1342219a1004181ac33ed8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240502 07:59:59.126807 26015 log.cc:1196] T 574fdd7a3c114ac9928ac12d1a7a4186 P f014c4ad4c1342219a1004181ac33ed8: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-7/wal/wals/574fdd7a3c114ac9928ac12d1a7a4186
I20240502 07:59:59.127292 25806 consensus_queue.cc:237] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { promote: false } }
I20240502 07:59:59.127377 19987 catalog_manager.cc:4830] TS f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): tablet 574fdd7a3c114ac9928ac12d1a7a4186 (table rebalance_test_table_6 [id=db1e20e82b894565a302e53bd7791607]) successfully deleted
I20240502 07:59:59.128679 20934 raft_consensus.cc:2932] T 574fdd7a3c114ac9928ac12d1a7a4186 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 07:59:59.129150 20273 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 LEARNER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 07:59:59.129717 26156 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:59.129810 22609 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 07:59:59.130134 20143 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 07:59:59.130168 25297 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:59.130625 25319 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:59.130746 22741 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 07:59:59.131304 20933 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 07:59:59.131644 25297 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:59.132508 20800 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 07:59:59.132869 20407 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 07:59:59.132957 25806 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:59.133226 25319 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:59.133386 25297 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:59.135929 26156 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [term 1 LEADER]: Committing config change with OpId 1.12: config changed from index 11 to 12, b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { promote: false } } }
I20240502 07:59:59.136358 22741 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { promote: false } } }
I20240502 07:59:59.137636 20800 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { promote: false } } }
I20240502 07:59:59.138245 20273 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { promote: false } } }
I20240502 07:59:59.138427 22609 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { promote: false } } }
I20240502 07:59:59.138552 20407 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { promote: false } } }
I20240502 07:59:59.140589 19999 catalog_manager.cc:5484] T 1bc5937510214c3da7731d3ad2a67842 P f014c4ad4c1342219a1004181ac33ed8 reported cstate change: config changed from index 11 to 12, b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b443e68b6212450286869124684bdfff" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { promote: false } } }
I20240502 07:59:59.144716 20143 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { promote: false } } }
I20240502 07:59:59.147341 25924 raft_consensus.cc:1050] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0: attempting to promote NON_VOTER e951f009258e473cb57cf8773c7975d4 to VOTER
I20240502 07:59:59.147500 20933 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, b7604c0eb3bf4a7cb260b564a94a1191 (127.3.139.66) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } attrs { replace: true } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { promote: false } } }
I20240502 07:59:59.147871 25924 consensus_queue.cc:237] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { promote: false } }
I20240502 07:59:59.149763 20537 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P e951f009258e473cb57cf8773c7975d4 [term 1 LEARNER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:59.151464 20407 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:59.151749 20143 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:59.151816 25924 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:59.151937 20668 consensus_queue.cc:237] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { promote: false } }
I20240502 07:59:59.152179 25924 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:59.152293 20273 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:59.152629 20671 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:59.152926 26172 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:59.153306 26172 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:59.153589 26172 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:59.153715 20800 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 07:59:59.154179 20406 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 07:59:59.154297 22593 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 07:59:59.154430 22741 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 07:59:59.154565 25297 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 07:59:59.154709 26156 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 07:59:59.154937 20934 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:59.154944 20273 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 07:59:59.155310 25297 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 07:59:59.155709 20933 raft_consensus.cc:1261] T 1bc5937510214c3da7731d3ad2a67842 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 07:59:59.155905 25924 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:59.156476 22609 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:59.157711 26159 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 07:59:59.157979 26156 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 07:59:59.158653 25924 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:59.159114 26159 consensus_queue.cc:1035] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 07:59:59.161598 26157 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff [term 1 LEADER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { promote: false } } }
I20240502 07:59:59.163239 20800 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { promote: false } } }
I20240502 07:59:59.163749 22593 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { promote: false } } }
I20240502 07:59:59.163848 20406 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { promote: false } } }
I20240502 07:59:59.164945 25093 raft_consensus.cc:1050] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4: attempting to promote NON_VOTER b443e68b6212450286869124684bdfff to VOTER
I20240502 07:59:59.165218 19987 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 1bc5937510214c3da7731d3ad2a67842 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:59.165469 25093 consensus_queue.cc:237] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } }
I20240502 07:59:59.165714 20934 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { promote: false } } }
I20240502 07:59:59.167299 20275 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { promote: false } } }
I20240502 07:59:59.168183 19992 catalog_manager.cc:5484] T 1bc5937510214c3da7731d3ad2a67842 P 35642aa2099c40e59bc2092604078e07 reported cstate change: config changed from index 12 to 13, VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) evicted. New cstate: current_term: 1 leader_uuid: "b443e68b6212450286869124684bdfff" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { promote: false } } }
I20240502 07:59:59.168344 20406 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:59.169144 20143 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:59.170261 22741 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:59.170341 20275 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:59.170300 20143 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e951f009258e473cb57cf8773c7975d4 (127.3.139.68) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { promote: false } } }
I20240502 07:59:59.170913 20668 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P b443e68b6212450286869124684bdfff [term 1 LEARNER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:59.171708 22741 raft_consensus.cc:2932] T 1bc5937510214c3da7731d3ad2a67842 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } attrs { promote: false } } }
I20240502 07:59:59.171761 20537 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e951f009258e473cb57cf8773c7975d4 (127.3.139.68) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { promote: false } } }
I20240502 07:59:59.172889 19997 catalog_manager.cc:5484] T 6a77bda800d44c4bbe1b082a5ea05b55 P 29a7180fe4574c09a8dce7c27500aab2 reported cstate change: config changed from index 8 to 9, e951f009258e473cb57cf8773c7975d4 (127.3.139.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c15e23a34059456788f9775031e27ec0" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { promote: false } } }
I20240502 07:59:59.174361 20933 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e951f009258e473cb57cf8773c7975d4 (127.3.139.68) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { promote: false } } }
I20240502 07:59:59.175145 25986 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:59.175557 25522 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:59.176158 20122 tablet_service.cc:1514] Processing DeleteTablet for tablet 1bc5937510214c3da7731d3ad2a67842 with delete_type TABLET_DATA_TOMBSTONED (TS 29a7180fe4574c09a8dce7c27500aab2 not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:44498
I20240502 07:59:59.176401 25633 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:59.176796 25984 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:59.177224 25985 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:59.177801 26204 tablet_replica.cc:330] T 1bc5937510214c3da7731d3ad2a67842 P 29a7180fe4574c09a8dce7c27500aab2: stopping tablet replica
I20240502 07:59:59.177965 26204 raft_consensus.cc:2229] T 1bc5937510214c3da7731d3ad2a67842 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:59.178285 26204 raft_consensus.cc:2258] T 1bc5937510214c3da7731d3ad2a67842 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:59.179255 26204 ts_tablet_manager.cc:1875] T 1bc5937510214c3da7731d3ad2a67842 P 29a7180fe4574c09a8dce7c27500aab2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:59.179379 20800 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:59.180233 26176 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e951f009258e473cb57cf8773c7975d4 (127.3.139.68) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { promote: false } } }
I20240502 07:59:59.180447 25983 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:59.182191 22609 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e951f009258e473cb57cf8773c7975d4 (127.3.139.68) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { promote: false } } }
I20240502 07:59:59.182404 26204 ts_tablet_manager.cc:1888] T 1bc5937510214c3da7731d3ad2a67842 P 29a7180fe4574c09a8dce7c27500aab2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.12
I20240502 07:59:59.182971 26204 log.cc:1196] T 1bc5937510214c3da7731d3ad2a67842 P 29a7180fe4574c09a8dce7c27500aab2: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-0/wal/wals/1bc5937510214c3da7731d3ad2a67842
I20240502 07:59:59.183264 26141 raft_consensus.cc:1050] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07: attempting to promote NON_VOTER f014c4ad4c1342219a1004181ac33ed8 to VOTER
I20240502 07:59:59.183738 26141 consensus_queue.cc:237] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } }
I20240502 07:59:59.185421 20934 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 07:59:59.185501 20537 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 07:59:59.185597 20801 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 07:59:59.185799 26142 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:59.186131 25926 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:59.186234 22741 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P f014c4ad4c1342219a1004181ac33ed8 [term 1 LEARNER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 07:59:59.186333 20275 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 07:59:59.187153 25926 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:59.187332 22475 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 07:59:59.187460 20272 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e951f009258e473cb57cf8773c7975d4 (127.3.139.68) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { promote: false } } }
I20240502 07:59:59.188108 25480 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:59.188163 20407 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e951f009258e473cb57cf8773c7975d4 (127.3.139.68) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { promote: false } } }
I20240502 07:59:59.188315 26142 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:59.188642 25480 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:59.189848 19986 catalog_manager.cc:4830] TS 29a7180fe4574c09a8dce7c27500aab2 (127.3.139.65:44703): tablet 1bc5937510214c3da7731d3ad2a67842 (table rebalance_test_table_1 [id=dd634821c68c405c9e104a7053619a61]) successfully deleted
I20240502 07:59:59.191428 20933 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 07:59:59.191761 22609 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 07:59:59.191812 26200 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:59.192407 25480 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:59.194181 25985 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, b443e68b6212450286869124684bdfff (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:59.195392 20668 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e951f009258e473cb57cf8773c7975d4 (127.3.139.68) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { promote: false } } }
I20240502 07:59:59.196528 20275 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, b443e68b6212450286869124684bdfff (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:59.196836 20671 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, b443e68b6212450286869124684bdfff (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:59.197968 22739 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, b443e68b6212450286869124684bdfff (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:59.201251 20407 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, b443e68b6212450286869124684bdfff (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:59.201485 19994 catalog_manager.cc:5484] T fb0f04b72dc046c0afee40b895e17a1d P b7604c0eb3bf4a7cb260b564a94a1191 reported cstate change: config changed from index 8 to 9, b443e68b6212450286869124684bdfff (127.3.139.69) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e951f009258e473cb57cf8773c7975d4" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:59.203547 20143 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, b443e68b6212450286869124684bdfff (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:59.204695 26142 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [term 1 LEADER]: Committing config change with OpId 1.12: config changed from index 11 to 12, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:59.205901 20801 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:59.207291 22609 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:59.207701 22741 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:59.209359 22475 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:59.209712 20934 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:59.210783 20537 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:59.213160 19997 catalog_manager.cc:5484] T e61fe3cb61c240f9b2c898aff1951144 P bfb1b3d596ef474faa229a835329cf20 reported cstate change: config changed from index 11 to 12, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "35642aa2099c40e59bc2092604078e07" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:59.213709 20933 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, b443e68b6212450286869124684bdfff (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:59.220750 20800 consensus_queue.cc:237] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { promote: false } }
I20240502 07:59:59.220985 20537 consensus_queue.cc:237] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } }
I20240502 07:59:59.221318 20407 consensus_queue.cc:237] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } }
I20240502 07:59:59.222333 20801 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, b443e68b6212450286869124684bdfff (127.3.139.69) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:59.223078 20671 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:59.223299 20933 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 07:59:59.223577 20143 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:59.223654 22609 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:59.223673 26143 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 07:59:59.224159 20275 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:59.224418 26174 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:59.225054 25984 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:59.225174 20275 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:59.225524 20671 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:59.226079 20275 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 07:59:59.226159 25984 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:59.226269 22739 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 07:59:59.226559 26143 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:59.226580 26178 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:59.226822 26143 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 07:59:59.226894 26178 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:59.227607 20273 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:59.228210 20407 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:59.228690 20140 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:59.229169 25984 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:59.230333 20801 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:59.230660 25984 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:59.230988 20537 raft_consensus.cc:1261] T 6a77bda800d44c4bbe1b082a5ea05b55 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer c15e23a34059456788f9775031e27ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:59.231025 26158 raft_consensus.cc:1050] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff: attempting to promote NON_VOTER c15e23a34059456788f9775031e27ec0 to VOTER
I20240502 07:59:59.231335 22475 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 07:59:59.231560 26158 consensus_queue.cc:237] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { promote: false } }
I20240502 07:59:59.232105 25480 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 07:59:59.232139 26172 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:59.232426 26172 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:59.233212 26200 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:59.233572 20407 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:59.233884 25522 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:59.234083 22741 raft_consensus.cc:1261] T fb0f04b72dc046c0afee40b895e17a1d P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Refusing update from remote peer e951f009258e473cb57cf8773c7975d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 07:59:59.234472 20538 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 07:59:59.234519 20275 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 07:59:59.234849 25319 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:59.234935 20140 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 07:59:59.235190 20802 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 07:59:59.235352 22593 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 07:59:59.235888 25806 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:59.235993 20934 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 07:59:59.236192 25806 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:59.236177 20140 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:59.236331 26159 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:59.238153 20801 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P c15e23a34059456788f9775031e27ec0 [term 1 LEARNER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 07:59:59.238235 20800 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:59.238636 25806 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:59.238916 25522 consensus_queue.cc:1035] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:59.239001 20272 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:59.239435 19999 catalog_manager.cc:5484] T fb0f04b72dc046c0afee40b895e17a1d P 29a7180fe4574c09a8dce7c27500aab2 reported cstate change: config changed from index 9 to 10, VOTER b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71) evicted. New cstate: current_term: 1 leader_uuid: "e951f009258e473cb57cf8773c7975d4" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:59.239766 20668 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:59.240999 19986 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet fb0f04b72dc046c0afee40b895e17a1d with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:59.241438 22740 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:59.242026 22741 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 07:59:59.242314 20406 raft_consensus.cc:2932] T fb0f04b72dc046c0afee40b895e17a1d P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } attrs { promote: false } } }
I20240502 07:59:59.243300 25480 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 07:59:59.243615 25924 consensus_queue.cc:1035] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:59.245239 25806 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:59.245467 20537 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 07:59:59.246282 20915 tablet_service.cc:1514] Processing DeleteTablet for tablet fb0f04b72dc046c0afee40b895e17a1d with delete_type TABLET_DATA_TOMBSTONED (TS b5f471f8b5944986a7d878cf6ac550b8 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:41530
I20240502 07:59:59.246505 26153 tablet_replica.cc:330] T fb0f04b72dc046c0afee40b895e17a1d P b5f471f8b5944986a7d878cf6ac550b8: stopping tablet replica
I20240502 07:59:59.246484 26143 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 07:59:59.246644 26153 raft_consensus.cc:2229] T fb0f04b72dc046c0afee40b895e17a1d P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:59.246793 26153 raft_consensus.cc:2258] T fb0f04b72dc046c0afee40b895e17a1d P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:59.247566 26153 ts_tablet_manager.cc:1875] T fb0f04b72dc046c0afee40b895e17a1d P b5f471f8b5944986a7d878cf6ac550b8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:59.249774 26153 ts_tablet_manager.cc:1888] T fb0f04b72dc046c0afee40b895e17a1d P b5f471f8b5944986a7d878cf6ac550b8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240502 07:59:59.249837 26153 log.cc:1196] T fb0f04b72dc046c0afee40b895e17a1d P b5f471f8b5944986a7d878cf6ac550b8: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/wal/wals/fb0f04b72dc046c0afee40b895e17a1d
I20240502 07:59:59.250425 19988 catalog_manager.cc:4830] TS b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577): tablet fb0f04b72dc046c0afee40b895e17a1d (table rebalance_test_table_6 [id=db1e20e82b894565a302e53bd7791607]) successfully deleted
I20240502 07:59:59.246083 26176 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { promote: false } } }
I20240502 07:59:59.251816 26159 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:59.251814 26143 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [term 1 LEADER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:59.253140 20143 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { promote: false } } }
I20240502 07:59:59.254842 19988 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet e61fe3cb61c240f9b2c898aff1951144 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:59.254698 20933 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:59.255627 19993 catalog_manager.cc:5484] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 reported cstate change: config changed from index 12 to 13, VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) evicted. New cstate: current_term: 1 leader_uuid: "35642aa2099c40e59bc2092604078e07" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 07:59:59.256736 20406 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { promote: false } } }
I20240502 07:59:59.257265 20537 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { promote: false } } }
I20240502 07:59:59.257315 19994 catalog_manager.cc:5484] T 6a77bda800d44c4bbe1b082a5ea05b55 P 29a7180fe4574c09a8dce7c27500aab2 reported cstate change: config changed from index 9 to 10, VOTER b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71) evicted. New cstate: current_term: 1 leader_uuid: "c15e23a34059456788f9775031e27ec0" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { promote: false } } }
I20240502 07:59:59.257968 22740 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:59.259161 19986 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 6a77bda800d44c4bbe1b082a5ea05b55 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:59.259318 22475 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:59.260298 20273 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { promote: false } } }
I20240502 07:59:59.260349 20272 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:59.262647 22609 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { promote: false } } }
I20240502 07:59:59.265396 20668 raft_consensus.cc:2932] T 6a77bda800d44c4bbe1b082a5ea05b55 P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } attrs { promote: false } } }
I20240502 07:59:59.268369 26159 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [term 1 LEADER]: Committing config change with OpId 1.12: config changed from index 11 to 12, c15e23a34059456788f9775031e27ec0 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { promote: false } } }
I20240502 07:59:59.269570 20801 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:59.271095 20537 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 07:59:59.272882 20915 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a77bda800d44c4bbe1b082a5ea05b55 with delete_type TABLET_DATA_TOMBSTONED (TS b5f471f8b5944986a7d878cf6ac550b8 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:41530
I20240502 07:59:59.273542 22593 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, c15e23a34059456788f9775031e27ec0 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { promote: false } } }
I20240502 07:59:59.274807 22740 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, c15e23a34059456788f9775031e27ec0 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { promote: false } } }
I20240502 07:59:59.275403 20275 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, c15e23a34059456788f9775031e27ec0 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { promote: false } } }
I20240502 07:59:59.275395 20539 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, c15e23a34059456788f9775031e27ec0 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { promote: false } } }
I20240502 07:59:59.277938 20934 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, c15e23a34059456788f9775031e27ec0 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { promote: false } } }
I20240502 07:59:59.278237 19998 catalog_manager.cc:5484] T bbae35bae8c047a6982e128ffb77ca86 P b7604c0eb3bf4a7cb260b564a94a1191 reported cstate change: config changed from index 11 to 12, c15e23a34059456788f9775031e27ec0 (127.3.139.70) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b443e68b6212450286869124684bdfff" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { promote: false } } }
I20240502 07:59:59.280000 20801 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, c15e23a34059456788f9775031e27ec0 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { promote: false } } }
I20240502 07:59:59.280289 26153 tablet_replica.cc:330] T 6a77bda800d44c4bbe1b082a5ea05b55 P b5f471f8b5944986a7d878cf6ac550b8: stopping tablet replica
I20240502 07:59:59.280417 26153 raft_consensus.cc:2229] T 6a77bda800d44c4bbe1b082a5ea05b55 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:59.280618 26153 raft_consensus.cc:2258] T 6a77bda800d44c4bbe1b082a5ea05b55 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:59.282182 26153 ts_tablet_manager.cc:1875] T 6a77bda800d44c4bbe1b082a5ea05b55 P b5f471f8b5944986a7d878cf6ac550b8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:59.282150 20142 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, c15e23a34059456788f9775031e27ec0 (127.3.139.70) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } attrs { replace: true } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { promote: false } } }
I20240502 07:59:59.283857 22588 tablet_service.cc:1514] Processing DeleteTablet for tablet e61fe3cb61c240f9b2c898aff1951144 with delete_type TABLET_DATA_TOMBSTONED (TS ff9f2ccccdfa48a1afbc0e371f41c662 not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:55562
I20240502 07:59:59.284255 26153 ts_tablet_manager.cc:1888] T 6a77bda800d44c4bbe1b082a5ea05b55 P b5f471f8b5944986a7d878cf6ac550b8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240502 07:59:59.284315 26153 log.cc:1196] T 6a77bda800d44c4bbe1b082a5ea05b55 P b5f471f8b5944986a7d878cf6ac550b8: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/wal/wals/6a77bda800d44c4bbe1b082a5ea05b55
I20240502 07:59:59.284392 26245 tablet_replica.cc:330] T e61fe3cb61c240f9b2c898aff1951144 P ff9f2ccccdfa48a1afbc0e371f41c662: stopping tablet replica
I20240502 07:59:59.284710 26245 raft_consensus.cc:2229] T e61fe3cb61c240f9b2c898aff1951144 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:59.284876 19988 catalog_manager.cc:4830] TS b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577): tablet 6a77bda800d44c4bbe1b082a5ea05b55 (table rebalance_test_table_5 [id=ed17eb489c6e490e9427e387589e51fe]) successfully deleted
I20240502 07:59:59.285037 26245 raft_consensus.cc:2258] T e61fe3cb61c240f9b2c898aff1951144 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:59.286998 26245 ts_tablet_manager.cc:1875] T e61fe3cb61c240f9b2c898aff1951144 P ff9f2ccccdfa48a1afbc0e371f41c662: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:59.288195 20668 consensus_queue.cc:237] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { promote: false } }
I20240502 07:59:59.289577 20801 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 07:59:59.290107 25806 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 07:59:59.290742 20275 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 07:59:59.290844 22593 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 07:59:59.290869 26245 ts_tablet_manager.cc:1888] T e61fe3cb61c240f9b2c898aff1951144 P ff9f2ccccdfa48a1afbc0e371f41c662: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.12
I20240502 07:59:59.290948 26245 log.cc:1196] T e61fe3cb61c240f9b2c898aff1951144 P ff9f2ccccdfa48a1afbc0e371f41c662: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-9/wal/wals/e61fe3cb61c240f9b2c898aff1951144
I20240502 07:59:59.291224 26155 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 07:59:59.291553 26155 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 07:59:59.291611 20539 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 07:59:59.292033 20142 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 07:59:59.292608 26156 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 07:59:59.292698 19987 catalog_manager.cc:4830] TS ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): tablet e61fe3cb61c240f9b2c898aff1951144 (table rebalance_test_table_3 [id=1388635dd26f4d3791ee92dfbf8d374a]) successfully deleted
I20240502 07:59:59.292917 26156 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 07:59:59.293932 22740 raft_consensus.cc:1261] T bbae35bae8c047a6982e128ffb77ca86 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Refusing update from remote peer b443e68b6212450286869124684bdfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 07:59:59.294368 26156 consensus_queue.cc:1035] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 07:59:59.295843 25297 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff [term 1 LEADER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { promote: false } } }
I20240502 07:59:59.296367 22593 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { promote: false } } }
I20240502 07:59:59.298288 20539 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { promote: false } } }
I20240502 07:59:59.299005 19987 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet bbae35bae8c047a6982e128ffb77ca86 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 07:59:59.299312 20001 catalog_manager.cc:5484] T bbae35bae8c047a6982e128ffb77ca86 P ff9f2ccccdfa48a1afbc0e371f41c662 reported cstate change: config changed from index 12 to 13, VOTER b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71) evicted. New cstate: current_term: 1 leader_uuid: "b443e68b6212450286869124684bdfff" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { promote: false } } }
I20240502 07:59:59.299810 20142 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { promote: false } } }
I20240502 07:59:59.301052 20275 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { promote: false } } }
I20240502 07:59:59.302306 20801 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { promote: false } } }
I20240502 07:59:59.306377 22739 raft_consensus.cc:2932] T bbae35bae8c047a6982e128ffb77ca86 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } attrs { promote: false } } }
I20240502 07:59:59.310698 20915 tablet_service.cc:1514] Processing DeleteTablet for tablet bbae35bae8c047a6982e128ffb77ca86 with delete_type TABLET_DATA_TOMBSTONED (TS b5f471f8b5944986a7d878cf6ac550b8 not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:41530
I20240502 07:59:59.310922 26153 tablet_replica.cc:330] T bbae35bae8c047a6982e128ffb77ca86 P b5f471f8b5944986a7d878cf6ac550b8: stopping tablet replica
I20240502 07:59:59.311128 26153 raft_consensus.cc:2229] T bbae35bae8c047a6982e128ffb77ca86 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:59.311415 26153 raft_consensus.cc:2258] T bbae35bae8c047a6982e128ffb77ca86 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:59.313498 26153 ts_tablet_manager.cc:1875] T bbae35bae8c047a6982e128ffb77ca86 P b5f471f8b5944986a7d878cf6ac550b8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:59.315604 26153 ts_tablet_manager.cc:1888] T bbae35bae8c047a6982e128ffb77ca86 P b5f471f8b5944986a7d878cf6ac550b8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.12
I20240502 07:59:59.315673 26153 log.cc:1196] T bbae35bae8c047a6982e128ffb77ca86 P b5f471f8b5944986a7d878cf6ac550b8: Deleting WAL directory at /tmp/dist-test-taskcXDMw5/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714636687578600-3629-0/raft_consensus-itest-cluster/ts-6/wal/wals/bbae35bae8c047a6982e128ffb77ca86
I20240502 07:59:59.316151 19988 catalog_manager.cc:4830] TS b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577): tablet bbae35bae8c047a6982e128ffb77ca86 (table rebalance_test_table_3 [id=1388635dd26f4d3791ee92dfbf8d374a]) successfully deleted
I20240502 07:59:59.386368 20915 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:59:59.391865 20386 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:59:59.397262 20253 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:59:59.397445 20122 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:59:59.420023 20517 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:59:59.431545 20781 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:59:59.431654 22456 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:59.434562 20649 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:59:59.439491 22721 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:59.439563 22588 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:59.552511 20386 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:59:59.553365 20122 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:59:59.554622 20915 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:59:59.562753 20649 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:59:59.567513 22456 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:59.570853 20517 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:59:59.570910 20781 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:59:59.576021 22588 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:59.614990 22721 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:59.616468 20253 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:59:59.801941 19997 ts_manager.cc:292] Set tserver state for bfb1b3d596ef474faa229a835329cf20 to MAINTENANCE_MODE
I20240502 08:00:00.002104 22456 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 08:00:00.002113 20386 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 08:00:00.003191 20122 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 08:00:00.010761 20253 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 08:00:00.011288 20781 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 08:00:00.026291 20517 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 08:00:00.027968 20915 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 08:00:00.039881 22588 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 08:00:00.044065 20649 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 08:00:00.044440 22721 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 08:00:00.175621 22456 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 08:00:00.175910 20122 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 08:00:00.175961 20386 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 08:00:00.176695 20915 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 08:00:00.182947 22721 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 08:00:00.187831 20781 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 08:00:00.200618 20517 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 08:00:00.209028 20253 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 08:00:00.231004 22588 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 08:00:00.234903 20649 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 08:00:00.364032 20386 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 08:00:00.364027 20122 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 08:00:00.364569 20649 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 08:00:00.365694 22456 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 08:00:00.380034 20781 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 08:00:00.391245 20915 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 08:00:00.403434 22721 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 08:00:00.407194 20253 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 08:00:00.427026 20517 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 08:00:00.429224 22588 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 08:00:00.461835 20406 consensus_queue.cc:237] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 1.13, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } }
I20240502 08:00:00.463541 22475 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240502 08:00:00.463543 20933 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240502 08:00:00.463675 20801 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240502 08:00:00.464030 26493 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240502 08:00:00.464238 26413 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240502 08:00:00.464356 26494 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240502 08:00:00.464488 20275 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240502 08:00:00.464557 20539 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240502 08:00:00.464910 22740 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240502 08:00:00.464932 26413 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240502 08:00:00.465222 26413 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240502 08:00:00.466030 26493 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240502 08:00:00.467506 26413 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [term 1 LEADER]: Committing config change with OpId 1.14: config changed from index 13 to 14. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 08:00:00.468253 22475 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 08:00:00.468429 22740 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 08:00:00.470320 19998 catalog_manager.cc:5484] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 reported cstate change: config changed from index 13 to 14. New cstate: current_term: 1 leader_uuid: "35642aa2099c40e59bc2092604078e07" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 08:00:00.470340 20801 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 08:00:00.470778 20933 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 08:00:00.471683 20539 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 08:00:00.474071 20275 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } }
I20240502 08:00:00.475194 20406 consensus_queue.cc:237] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 1.14, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } }
I20240502 08:00:00.477885 22475 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240502 08:00:00.478193 20801 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240502 08:00:00.478251 26494 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240502 08:00:00.478536 20933 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240502 08:00:00.478665 22740 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
W20240502 08:00:00.479077 20342 consensus_peers.cc:485] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240502 08:00:00.479252 20539 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240502 08:00:00.479241 26495 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240502 08:00:00.479426 26492 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240502 08:00:00.479732 26493 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240502 08:00:00.480669 20275 raft_consensus.cc:1261] T e61fe3cb61c240f9b2c898aff1951144 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Refusing update from remote peer 35642aa2099c40e59bc2092604078e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240502 08:00:00.481051 26413 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240502 08:00:00.481622 26492 consensus_queue.cc:1035] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240502 08:00:00.482476 26413 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 [term 1 LEADER]: Committing config change with OpId 1.15: config changed from index 14 to 15, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 08:00:00.482980 22475 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 08:00:00.483665 20539 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 08:00:00.484094 19988 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet e61fe3cb61c240f9b2c898aff1951144 with cas_config_opid_index 14: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240502 08:00:00.484205 20933 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 08:00:00.484889 20801 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 08:00:00.485466 19993 catalog_manager.cc:5484] T e61fe3cb61c240f9b2c898aff1951144 P bfb1b3d596ef474faa229a835329cf20 reported cstate change: config changed from index 14 to 15, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New cstate: current_term: 1 leader_uuid: "35642aa2099c40e59bc2092604078e07" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 08:00:00.487484 22740 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 08:00:00.490658 20275 raft_consensus.cc:2932] T e61fe3cb61c240f9b2c898aff1951144 P b7604c0eb3bf4a7cb260b564a94a1191 [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "b7604c0eb3bf4a7cb260b564a94a1191" member_type: VOTER last_known_addr { host: "127.3.139.66" port: 45331 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 08:00:00.495817 20933 consensus_queue.cc:237] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 1.13, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: false } }
I20240502 08:00:00.497290 22740 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240502 08:00:00.497349 22475 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240502 08:00:00.497592 20142 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240502 08:00:00.497709 20406 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240502 08:00:00.497694 26517 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240502 08:00:00.498014 20801 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240502 08:00:00.498129 20539 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240502 08:00:00.498569 26114 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240502 08:00:00.498708 26515 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240502 08:00:00.498860 26514 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240502 08:00:00.499202 26519 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240502 08:00:00.499770 26516 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240502 08:00:00.501716 26114 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 LEADER]: Committing config change with OpId 1.14: config changed from index 13 to 14. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: false } } }
I20240502 08:00:00.502441 22475 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: false } } }
I20240502 08:00:00.502588 20143 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: false } } }
I20240502 08:00:00.504523 22740 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: false } } }
I20240502 08:00:00.505811 19997 catalog_manager.cc:5484] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 reported cstate change: config changed from index 13 to 14. New cstate: current_term: 1 leader_uuid: "b5f471f8b5944986a7d878cf6ac550b8" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 08:00:00.507815 20539 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: false } } }
I20240502 08:00:00.507819 20801 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: false } } }
I20240502 08:00:00.511647 20933 consensus_queue.cc:237] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 1.14, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } }
I20240502 08:00:00.513576 20406 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240502 08:00:00.513897 22475 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240502 08:00:00.513937 26517 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
W20240502 08:00:00.514145 20870 consensus_peers.cc:485] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f588a3ec49a24b1a9a9e19cdf8c30ca5. This is attempt 1: this message will repeat every 5th retry.
I20240502 08:00:00.514299 26517 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240502 08:00:00.514720 22740 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240502 08:00:00.515174 20801 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240502 08:00:00.515019 20406 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: false } } }
I20240502 08:00:00.515341 26514 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240502 08:00:00.516166 26517 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240502 08:00:00.516834 20142 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240502 08:00:00.517485 20539 raft_consensus.cc:1261] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240502 08:00:00.517808 26516 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240502 08:00:00.518352 26526 consensus_queue.cc:1035] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240502 08:00:00.519510 26514 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 [term 1 LEADER]: Committing config change with OpId 1.15: config changed from index 14 to 15, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 08:00:00.520131 20142 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 08:00:00.520895 20539 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 08:00:00.522109 22740 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P f014c4ad4c1342219a1004181ac33ed8 [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 08:00:00.522501 19988 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet f588a3ec49a24b1a9a9e19cdf8c30ca5 with cas_config_opid_index 14: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240502 08:00:00.522755 20800 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P c15e23a34059456788f9775031e27ec0 [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 08:00:00.523082 20406 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 08:00:00.525344 20000 catalog_manager.cc:5484] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 reported cstate change: config changed from index 14 to 15, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New cstate: current_term: 1 leader_uuid: "b5f471f8b5944986a7d878cf6ac550b8" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 08:00:00.525766 22475 raft_consensus.cc:2932] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, NON_VOTER ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "c15e23a34059456788f9775031e27ec0" member_type: VOTER last_known_addr { host: "127.3.139.70" port: 36085 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: false } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } attrs { promote: false } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: NON_VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: true } } }
I20240502 08:00:00.532835 20933 consensus_queue.cc:237] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } }
I20240502 08:00:00.534541 22475 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 08:00:00.534541 20406 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 08:00:00.534639 20671 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 08:00:00.534893 20539 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 08:00:00.534943 20142 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 08:00:00.534919 26519 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 08:00:00.535490 26517 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 08:00:00.535645 26516 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 08:00:00.535802 26114 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 08:00:00.536335 22593 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 08:00:00.536784 26519 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 08:00:00.537938 26514 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 08:00:00.538676 26517 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 10 to 11. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 08:00:00.539086 22593 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 08:00:00.539865 20142 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 08:00:00.540815 20539 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 08:00:00.540989 22476 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 08:00:00.542035 20671 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 08:00:00.542584 20406 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } }
I20240502 08:00:00.543325 20001 catalog_manager.cc:5484] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 reported cstate change: config changed from index 10 to 11. New cstate: current_term: 1 leader_uuid: "b5f471f8b5944986a7d878cf6ac550b8" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 08:00:00.548007 20933 consensus_queue.cc:237] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } }
I20240502 08:00:00.549759 20539 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 08:00:00.549780 22593 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 08:00:00.549780 20142 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 08:00:00.550076 20406 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 08:00:00.550137 26526 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 08:00:00.550328 26114 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 08:00:00.550410 26526 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 08:00:00.551162 22475 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 08:00:00.551625 20671 raft_consensus.cc:1261] T 1453e833d0ad494583c44b5496088f6d P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Refusing update from remote peer b5f471f8b5944986a7d878cf6ac550b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 08:00:00.550650 26517 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20240502 08:00:00.552289 20870 consensus_peers.cc:485] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1453e833d0ad494583c44b5496088f6d. This is attempt 1: this message will repeat every 5th retry.
I20240502 08:00:00.552520 26516 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 08:00:00.553535 26514 consensus_queue.cc:1035] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 08:00:00.554965 26114 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 [term 1 LEADER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 08:00:00.555603 20539 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P e951f009258e473cb57cf8773c7975d4 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 08:00:00.555884 22475 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P bfb1b3d596ef474faa229a835329cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 08:00:00.557345 20406 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P 35642aa2099c40e59bc2092604078e07 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 08:00:00.557749 19995 catalog_manager.cc:5484] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 reported cstate change: config changed from index 11 to 12, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New cstate: current_term: 1 leader_uuid: "b5f471f8b5944986a7d878cf6ac550b8" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 08:00:00.558264 22593 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P ff9f2ccccdfa48a1afbc0e371f41c662 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 08:00:00.558480 19988 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 1453e833d0ad494583c44b5496088f6d with cas_config_opid_index 11: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240502 08:00:00.559993 20671 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P b443e68b6212450286869124684bdfff [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 08:00:00.563165 20142 raft_consensus.cc:2932] T 1453e833d0ad494583c44b5496088f6d P 29a7180fe4574c09a8dce7c27500aab2 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "29a7180fe4574c09a8dce7c27500aab2" member_type: VOTER last_known_addr { host: "127.3.139.65" port: 44703 } } peers { permanent_uuid: "35642aa2099c40e59bc2092604078e07" member_type: VOTER last_known_addr { host: "127.3.139.67" port: 45587 } } peers { permanent_uuid: "e951f009258e473cb57cf8773c7975d4" member_type: VOTER last_known_addr { host: "127.3.139.68" port: 40315 } } peers { permanent_uuid: "b443e68b6212450286869124684bdfff" member_type: VOTER last_known_addr { host: "127.3.139.69" port: 37923 } } peers { permanent_uuid: "b5f471f8b5944986a7d878cf6ac550b8" member_type: VOTER last_known_addr { host: "127.3.139.71" port: 39577 } } peers { permanent_uuid: "bfb1b3d596ef474faa229a835329cf20" member_type: VOTER last_known_addr { host: "127.3.139.73" port: 38741 } attrs { promote: false replace: true } } peers { permanent_uuid: "ff9f2ccccdfa48a1afbc0e371f41c662" member_type: VOTER last_known_addr { host: "127.3.139.74" port: 45559 } attrs { promote: false } } peers { permanent_uuid: "f014c4ad4c1342219a1004181ac33ed8" member_type: NON_VOTER last_known_addr { host: "127.3.139.72" port: 43861 } attrs { promote: true } } }
I20240502 08:00:00.966912 26571 ts_tablet_manager.cc:902] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P ff9f2ccccdfa48a1afbc0e371f41c662: Initiating tablet copy from peer b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577)
F20240502 08:00:00.967139 26571 trace_metrics.cc:68] Check failed: g_intern_map->size() < 100 (100 vs. 100) Too many interned strings: (MaintenanceMgr.queue_time_us, MaintenanceMgr.queue_time_us) (MaintenanceMgr.run_cpu_time_us, MaintenanceMgr.run_cpu_time_us) (MaintenanceMgr.run_wall_time_us, MaintenanceMgr.run_wall_time_us) (apply.queue_time_us, apply.queue_time_us) (apply.run_cpu_time_us, apply.run_cpu_time_us) (apply.run_wall_time_us, apply.run_wall_time_us) (client-negotiator.queue_time_us, client-negotiator.queue_time_us) (client-negotiator.run_cpu_time_us, client-negotiator.run_cpu_time_us) (client-negotiator.run_wall_time_us, client-negotiator.run_wall_time_us) (compiler_manager_pool.queue_time_us, compiler_manager_pool.queue_time_us) (compiler_manager_pool.run_cpu_time_us, compiler_manager_pool.run_cpu_time_us) (compiler_manager_pool.run_wall_time_us, compiler_manager_pool.run_wall_time_us) (dirs.queue_time_us, dirs.queue_time_us) (dirs.run_cpu_time_us, dirs.run_cpu_time_us) (dirs.run_wall_time_us, dirs.run_wall_time_us) (dns-resolver.queue_time_us, dns-resolver.queue_time_us) (dns-resolver.run_cpu_time_us, dns-resolver.run_cpu_time_us) (dns-resolver.run_wall_time_us, dns-resolver.run_wall_time_us) (log-alloc.queue_time_us, log-alloc.queue_time_us) (log-alloc.run_cpu_time_us, log-alloc.run_cpu_time_us) (log-alloc.run_wall_time_us, log-alloc.run_wall_time_us) (prepare.queue_time_us, prepare.queue_time_us) (prepare.run_cpu_time_us, prepare.run_cpu_time_us) (prepare.run_wall_time_us, prepare.run_wall_time_us) (raft.queue_time_us, raft.queue_time_us) (raft.run_cpu_time_us, raft.run_cpu_time_us) (raft.run_wall_time_us, raft.run_wall_time_us) (repair_pool.queue_time_us, repair_pool.queue_time_us) (repair_pool.run_cpu_time_us, repair_pool.run_cpu_time_us) (repair_pool.run_wall_time_us, repair_pool.run_wall_time_us) (server-negotiator.queue_time_us, server-negotiator.queue_time_us) (server-negotiator.run_cpu_time_us, server-negotiator.run_cpu_time_us) (server-negotiator.run_wall_time_us, server-negotiator.run_wall_time_us) (tablet-copy.queue_time_us, tablet-copy.queue_time_us) (tablet-copy.run_cpu_time_us, tablet-copy.run_cpu_time_us) (tablet-copy.run_wall_time_us, tablet-copy.run_wall_time_us) (tablet-delete.queue_time_us, tablet-delete.queue_time_us) (tablet-delete.run_cpu_time_us, tablet-delete.run_cpu_time_us) (tablet-delete.run_wall_time_us, tablet-delete.run_wall_time_us) (tablet-download-pool-1453e833d0ad494583c44b5496088f6d.queue_time_us, tablet-download-pool-1453e833d0ad494583c44b5496088f6d.queue_time_us) (tablet-download-pool-1453e833d0ad494583c44b5496088f6d.run_cpu_time_us, tablet-download-pool-1453e833d0ad494583c44b5496088f6d.run_cpu_time_us) (tablet-download-pool-1453e833d0ad494583c44b5496088f6d.run_wall_time_us, tablet-download-pool-1453e833d0ad494583c44b5496088f6d.run_wall_time_us) (tablet-download-pool-15bb9480e88b41219cc91168cc438c70.queue_time_us, tablet-download-pool-15bb9480e88b41219cc91168cc438c70.queue_time_us) (tablet-download-pool-15bb9480e88b41219cc91168cc438c70.run_cpu_time_us, tablet-download-pool-15bb9480e88b41219cc91168cc438c70.run_cpu_time_us) (tablet-download-pool-15bb9480e88b41219cc91168cc438c70.run_wall_time_us, tablet-download-pool-15bb9480e88b41219cc91168cc438c70.run_wall_time_us) (tablet-download-pool-1bc5937510214c3da7731d3ad2a67842.queue_time_us, tablet-download-pool-1bc5937510214c3da7731d3ad2a67842.queue_time_us) (tablet-download-pool-1bc5937510214c3da7731d3ad2a67842.run_cpu_time_us, tablet-download-pool-1bc5937510214c3da7731d3ad2a67842.run_cpu_time_us) (tablet-download-pool-1bc5937510214c3da7731d3ad2a67842.run_wall_time_us, tablet-download-pool-1bc5937510214c3da7731d3ad2a67842.run_wall_time_us) (tablet-download-pool-574fdd7a3c114ac9928ac12d1a7a4186.queue_time_us, tablet-download-pool-574fdd7a3c114ac9928ac12d1a7a4186.queue_time_us) (tablet-download-pool-574fdd7a3c114ac9928ac12d1a7a4186.run_cpu_time_us, tablet-download-pool-574fdd7a3c114ac9928ac12d1a7a4186.run_cpu_time_us) (tablet-download-pool-574fdd7a3c114ac9928ac12d1a7a4186.run_wall_time_us, tablet-download-pool-574fdd7a3c114ac9928ac12d1a7a4186.run_wall_time_us) (tablet-download-pool-6a77bda800d44c4bbe1b082a5ea05b55.queue_time_us, tablet-download-pool-6a77bda800d44c4bbe1b082a5ea05b55.queue_time_us) (tablet-download-pool-6a77bda800d44c4bbe1b082a5ea05b55.run_cpu_time_us, tablet-download-pool-6a77bda800d44c4bbe1b082a5ea05b55.run_cpu_time_us) (tablet-download-pool-6a77bda800d44c4bbe1b082a5ea05b55.run_wall_time_us, tablet-download-pool-6a77bda800d44c4bbe1b082a5ea05b55.run_wall_time_us) (tablet-download-pool-880876d792554606a7ccf21f9ec08e0f.queue_time_us, tablet-download-pool-880876d792554606a7ccf21f9ec08e0f.queue_time_us) (tablet-download-pool-880876d792554606a7ccf21f9ec08e0f.run_cpu_time_us, tablet-download-pool-880876d792554606a7ccf21f9ec08e0f.run_cpu_time_us) (tablet-download-pool-880876d792554606a7ccf21f9ec08e0f.run_wall_time_us, tablet-download-pool-880876d792554606a7ccf21f9ec08e0f.run_wall_time_us) (tablet-download-pool-9ad40762dc784dfa9f3507e025db32b8.queue_time_us, tablet-download-pool-9ad40762dc784dfa9f3507e025db32b8.queue_time_us) (tablet-download-pool-9ad40762dc784dfa9f3507e025db32b8.run_cpu_time_us, tablet-download-pool-9ad40762dc784dfa9f3507e025db32b8.run_cpu_time_us) (tablet-download-pool-9ad40762dc784dfa9f3507e025db32b8.run_wall_time_us, tablet-download-pool-9ad40762dc784dfa9f3507e025db32b8.run_wall_time_us) (tablet-download-pool-a973de06330c487ba54a8a36b1e31b54.queue_time_us, tablet-download-pool-a973de06330c487ba54a8a36b1e31b54.queue_time_us) (tablet-download-pool-a973de06330c487ba54a8a36b1e31b54.run_cpu_time_us, tablet-download-pool-a973de06330c487ba54a8a36b1e31b54.run_cpu_time_us) (tablet-download-pool-a973de06330c487ba54a8a36b1e31b54.run_wall_time_us, tablet-download-pool-a973de06330c487ba54a8a36b1e31b54.run_wall_time_us) (tablet-download-pool-ae6d597fbc9242539edee8685a015d99.queue_time_us, tablet-download-pool-ae6d597fbc9242539edee8685a015d99.queue_time_us) (tablet-download-pool-ae6d597fbc9242539edee8685a015d99.run_cpu_time_us, tablet-download-pool-ae6d597fbc9242539edee8685a015d99.run_cpu_time_us) (tablet-download-pool-ae6d597fbc9242539edee8685a015d99.run_wall_time_us, tablet-download-pool-ae6d597fbc9242539edee8685a015d99.run_wall_time_us) (tablet-download-pool-b73e0ce533014420af513429620af68c.queue_time_us, tablet-download-pool-b73e0ce533014420af513429620af68c.queue_time_us) (tablet-download-pool-b73e0ce533014420af513429620af68c.run_cpu_time_us, tablet-download-pool-b73e0ce533014420af513429620af68c.run_cpu_time_us) (tablet-download-pool-b73e0ce533014420af513429620af68c.run_wall_time_us, tablet-download-pool-b73e0ce533014420af513429620af68c.run_wall_time_us) (tablet-download-pool-bbae35bae8c047a6982e128ffb77ca86.queue_time_us, tablet-download-pool-bbae35bae8c047a6982e128ffb77ca86.queue_time_us) (tablet-download-pool-bbae35bae8c047a6982e128ffb77ca86.run_cpu_time_us, tablet-download-pool-bbae35bae8c047a6982e128ffb77ca86.run_cpu_time_us) (tablet-download-pool-bbae35bae8c047a6982e128ffb77ca86.run_wall_time_us, tablet-download-pool-bbae35bae8c047a6982e128ffb77ca86.run_wall_time_us) (tablet-download-pool-c3ca6f24c92d4273ac6c9d83161813d7.queue_time_us, tablet-download-pool-c3ca6f24c92d4273ac6c9d83161813d7.queue_time_us) (tablet-download-pool-c3ca6f24c92d4273ac6c9d83161813d7.run_cpu_time_us, tablet-download-pool-c3ca6f24c92d4273ac6c9d83161813d7.run_cpu_time_us) (tablet-download-pool-c3ca6f24c92d4273ac6c9d83161813d7.run_wall_time_us, tablet-download-pool-c3ca6f24c92d4273ac6c9d83161813d7.run_wall_time_us) (tablet-download-pool-dbf4b90cac9e4ff3b2936a1fa2ca8561.queue_time_us, tablet-download-pool-dbf4b90cac9e4ff3b2936a1fa2ca8561.queue_time_us) (tablet-download-pool-dbf4b90cac9e4ff3b2936a1fa2ca8561.run_cpu_time_us, tablet-download-pool-dbf4b90cac9e4ff3b2936a1fa2ca8561.run_cpu_time_us) (tablet-download-pool-dbf4b90cac9e4ff3b2936a1fa2ca8561.run_wall_time_us, tablet-download-pool-dbf4b90cac9e4ff3b2936a1fa2ca8561.run_wall_time_us) (tablet-download-pool-e61fe3cb61c240f9b2c898aff1951144.queue_time_us, tablet-download-pool-e61fe3cb61c240f9b2c898aff1951144.queue_time_us) (tablet-download-pool-e61fe3cb61c240f9b2c898aff1951144.run_cpu_time_us, tablet-download-pool-e61fe3cb61c240f9b2c898aff1951144.run_cpu_time_us) (tablet-download-pool-e61fe3cb61c240f9b2c898aff1951144.run_wall_time_us, tablet-download-pool-e61fe3cb61c240f9b2c898aff1951144.run_wall_time_us) (tablet-download-pool-f588a3ec49a24b1a9a9e19cdf8c30ca5.queue_time_us, tablet-download-pool-f588a3ec49a24b1a9a9e19cdf8c30ca5.queue_time_us) (tablet-open.queue_time_us, tablet-open.queue_time_us) (tablet-open.run_cpu_time_us, tablet-open.run_cpu_time_us) (tablet-open.run_wall_time_us, tablet-open.run_wall_time_us) (txn-commit.queue_time_us, txn-commit.queue_time_us) (txn-commit.run_cpu_time_us, txn-commit.run_cpu_time_us) (txn-commit.run_wall_time_us, txn-commit.run_wall_time_us) (txn-participant-registration.queue_time_us, txn-participant-registration.queue_time_us) (txn-participant-registration.run_cpu_time_us, txn-participant-registration.run_cpu_time_us) (txn-participant-registration.run_wall_time_us, txn-participant-registration.run_wall_time_us) (txn-status-manager.queue_time_us, txn-status-manager.queue_time_us) (txn-status-manager.run_cpu_time_us, txn-status-manager.run_cpu_time_us) (txn-status-manager.run_wall_time_us, txn-status-manager.run_wall_time_us) (txn-status-tablet-reload.queue_time_us, txn-status-tablet-reload.queue_time_us) (txn-status-tablet-reload.run_cpu_time_us, txn-status-tablet-reload.run_cpu_time_us) (txn-status-tablet-reload.run_wall_time_us, txn-status-tablet-reload.run_wall_time_us) (wal-append.queue_time_us, wal-append.queue_time_us) (wal-append.run_cpu_time_us, wal-append.run_cpu_time_us) (wal-append.run_wall_time_us, wal-append.run_wall_time_us)
*** Check failure stack trace: ***
@ 0x7f741b8dadcd google::LogMessage::Fail() at ??:0
@ 0x7f741b8deb93 google::LogMessage::SendToLog() at ??:0
@ 0x7f741b8da7cc google::LogMessage::Flush() at ??:0
@ 0x7f741b8dbf59 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f741cd44044 kudu::TraceMetrics::InternName() at ??:0
@ 0x7f741cd28f98 kudu::ThreadPool::ThreadPool() at ??:0
@ 0x7f741cd26efc kudu::ThreadPoolBuilder::Build() at ??:0
@ 0x7f741f8d977a kudu::tserver::TabletCopyClient::TabletCopyClient() at ??:0
@ 0x7f741f8e2a45 kudu::tserver::RemoteTabletCopyClient::RemoteTabletCopyClient() at ??:0
@ 0x7f741f96376f kudu::tserver::TSTabletManager::RunTabletCopy() at ??:0
@ 0x7f741f9774f6 kudu::tserver::TabletCopyRunnable::Run() at ??:0
@ 0x7f741f962226 _ZZN4kudu7tserver15TSTabletManager15StartTabletCopyEPKNS_9consensus24StartTabletCopyRequestPBESt8functionIFvRKNS_6StatusENS0_24TabletServerErrorPB_CodeEEEENKUlvE_clEv at ??:0
@ 0x7f741f971c0a _ZNSt17_Function_handlerIFvvEZN4kudu7tserver15TSTabletManager15StartTabletCopyEPKNS1_9consensus24StartTabletCopyRequestPBESt8functionIFvRKNS1_6StatusENS2_24TabletServerErrorPB_CodeEEEEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x562fb4bcc75c std::function<>::operator()() at ??:0
@ 0x7f741cd2be06 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f741cd2c6ef _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f741cd2e03b _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x562fb4bcc75c std::function<>::operator()() at ??:0
@ 0x7f741cd1b807 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f741d5596db start_thread at ??:0
@ 0x7f741aa9171f clone at ??:0
I20240502 08:00:01.117626 26610 ts_tablet_manager.cc:902] T 1453e833d0ad494583c44b5496088f6d P f014c4ad4c1342219a1004181ac33ed8: Initiating tablet copy from peer b5f471f8b5944986a7d878cf6ac550b8 (127.3.139.71:39577)
F20240502 08:00:01.117830 26610 trace_metrics.cc:68] Check failed: g_intern_map->size() < 100 (100 vs. 100) Too many interned strings: (MaintenanceMgr.queue_time_us, MaintenanceMgr.queue_time_us) (MaintenanceMgr.run_cpu_time_us, MaintenanceMgr.run_cpu_time_us) (MaintenanceMgr.run_wall_time_us, MaintenanceMgr.run_wall_time_us) (apply.queue_time_us, apply.queue_time_us) (apply.run_cpu_time_us, apply.run_cpu_time_us) (apply.run_wall_time_us, apply.run_wall_time_us) (client-negotiator.queue_time_us, client-negotiator.queue_time_us) (client-negotiator.run_cpu_time_us, client-negotiator.run_cpu_time_us) (client-negotiator.run_wall_time_us, client-negotiator.run_wall_time_us) (compiler_manager_pool.queue_time_us, compiler_manager_pool.queue_time_us) (compiler_manager_pool.run_cpu_time_us, compiler_manager_pool.run_cpu_time_us) (compiler_manager_pool.run_wall_time_us, compiler_manager_pool.run_wall_time_us) (dirs.queue_time_us, dirs.queue_time_us) (dirs.run_cpu_time_us, dirs.run_cpu_time_us) (dirs.run_wall_time_us, dirs.run_wall_time_us) (dns-resolver.queue_time_us, dns-resolver.queue_time_us) (dns-resolver.run_cpu_time_us, dns-resolver.run_cpu_time_us) (dns-resolver.run_wall_time_us, dns-resolver.run_wall_time_us) (log-alloc.queue_time_us, log-alloc.queue_time_us) (log-alloc.run_cpu_time_us, log-alloc.run_cpu_time_us) (log-alloc.run_wall_time_us, log-alloc.run_wall_time_us) (prepare.queue_time_us, prepare.queue_time_us) (prepare.run_cpu_time_us, prepare.run_cpu_time_us) (prepare.run_wall_time_us, prepare.run_wall_time_us) (raft.queue_time_us, raft.queue_time_us) (raft.run_cpu_time_us, raft.run_cpu_time_us) (raft.run_wall_time_us, raft.run_wall_time_us) (repair_pool.queue_time_us, repair_pool.queue_time_us) (repair_pool.run_cpu_time_us, repair_pool.run_cpu_time_us) (repair_pool.run_wall_time_us, repair_pool.run_wall_time_us) (server-negotiator.queue_time_us, server-negotiator.queue_time_us) (server-negotiator.run_cpu_time_us, server-negotiator.run_cpu_time_us) (server-negotiator.run_wall_time_us, server-negotiator.run_wall_time_us) (tablet-copy.queue_time_us, tablet-copy.queue_time_us) (tablet-copy.run_cpu_time_us, tablet-copy.run_cpu_time_us) (tablet-copy.run_wall_time_us, tablet-copy.run_wall_time_us) (tablet-delete.queue_time_us, tablet-delete.queue_time_us) (tablet-delete.run_cpu_time_us, tablet-delete.run_cpu_time_us) (tablet-delete.run_wall_time_us, tablet-delete.run_wall_time_us) (tablet-download-pool-04659b5e3504411987dafc49a5fa5ea7.queue_time_us, tablet-download-pool-04659b5e3504411987dafc49a5fa5ea7.queue_time_us) (tablet-download-pool-04659b5e3504411987dafc49a5fa5ea7.run_cpu_time_us, tablet-download-pool-04659b5e3504411987dafc49a5fa5ea7.run_cpu_time_us) (tablet-download-pool-04659b5e3504411987dafc49a5fa5ea7.run_wall_time_us, tablet-download-pool-04659b5e3504411987dafc49a5fa5ea7.run_wall_time_us) (tablet-download-pool-0e2908de10684c84af542f16ff8727cb.queue_time_us, tablet-download-pool-0e2908de10684c84af542f16ff8727cb.queue_time_us) (tablet-download-pool-0e2908de10684c84af542f16ff8727cb.run_cpu_time_us, tablet-download-pool-0e2908de10684c84af542f16ff8727cb.run_cpu_time_us) (tablet-download-pool-0e2908de10684c84af542f16ff8727cb.run_wall_time_us, tablet-download-pool-0e2908de10684c84af542f16ff8727cb.run_wall_time_us) (tablet-download-pool-0f03a9b07a8c40bdbdf3ff714302a3ea.queue_time_us, tablet-download-pool-0f03a9b07a8c40bdbdf3ff714302a3ea.queue_time_us) (tablet-download-pool-0f03a9b07a8c40bdbdf3ff714302a3ea.run_cpu_time_us, tablet-download-pool-0f03a9b07a8c40bdbdf3ff714302a3ea.run_cpu_time_us) (tablet-download-pool-0f03a9b07a8c40bdbdf3ff714302a3ea.run_wall_time_us, tablet-download-pool-0f03a9b07a8c40bdbdf3ff714302a3ea.run_wall_time_us) (tablet-download-pool-1453e833d0ad494583c44b5496088f6d.queue_time_us, tablet-download-pool-1453e833d0ad494583c44b5496088f6d.queue_time_us) (tablet-download-pool-15bb9480e88b41219cc91168cc438c70.queue_time_us, tablet-download-pool-15bb9480e88b41219cc91168cc438c70.queue_time_us) (tablet-download-pool-15bb9480e88b41219cc91168cc438c70.run_cpu_time_us, tablet-download-pool-15bb9480e88b41219cc91168cc438c70.run_cpu_time_us) (tablet-download-pool-15bb9480e88b41219cc91168cc438c70.run_wall_time_us, tablet-download-pool-15bb9480e88b41219cc91168cc438c70.run_wall_time_us) (tablet-download-pool-1bc5937510214c3da7731d3ad2a67842.queue_time_us, tablet-download-pool-1bc5937510214c3da7731d3ad2a67842.queue_time_us) (tablet-download-pool-1bc5937510214c3da7731d3ad2a67842.run_cpu_time_us, tablet-download-pool-1bc5937510214c3da7731d3ad2a67842.run_cpu_time_us) (tablet-download-pool-1bc5937510214c3da7731d3ad2a67842.run_wall_time_us, tablet-download-pool-1bc5937510214c3da7731d3ad2a67842.run_wall_time_us) (tablet-download-pool-331368e52904494fa5ecf1ef594821fc.queue_time_us, tablet-download-pool-331368e52904494fa5ecf1ef594821fc.queue_time_us) (tablet-download-pool-331368e52904494fa5ecf1ef594821fc.run_cpu_time_us, tablet-download-pool-331368e52904494fa5ecf1ef594821fc.run_cpu_time_us) (tablet-download-pool-331368e52904494fa5ecf1ef594821fc.run_wall_time_us, tablet-download-pool-331368e52904494fa5ecf1ef594821fc.run_wall_time_us) (tablet-download-pool-574fdd7a3c114ac9928ac12d1a7a4186.queue_time_us, tablet-download-pool-574fdd7a3c114ac9928ac12d1a7a4186.queue_time_us) (tablet-download-pool-574fdd7a3c114ac9928ac12d1a7a4186.run_cpu_time_us, tablet-download-pool-574fdd7a3c114ac9928ac12d1a7a4186.run_cpu_time_us) (tablet-download-pool-574fdd7a3c114ac9928ac12d1a7a4186.run_wall_time_us, tablet-download-pool-574fdd7a3c114ac9928ac12d1a7a4186.run_wall_time_us) (tablet-download-pool-9ad40762dc784dfa9f3507e025db32b8.queue_time_us, tablet-download-pool-9ad40762dc784dfa9f3507e025db32b8.queue_time_us) (tablet-download-pool-9ad40762dc784dfa9f3507e025db32b8.run_cpu_time_us, tablet-download-pool-9ad40762dc784dfa9f3507e025db32b8.run_cpu_time_us) (tablet-download-pool-9ad40762dc784dfa9f3507e025db32b8.run_wall_time_us, tablet-download-pool-9ad40762dc784dfa9f3507e025db32b8.run_wall_time_us) (tablet-download-pool-a973de06330c487ba54a8a36b1e31b54.queue_time_us, tablet-download-pool-a973de06330c487ba54a8a36b1e31b54.queue_time_us) (tablet-download-pool-a973de06330c487ba54a8a36b1e31b54.run_cpu_time_us, tablet-download-pool-a973de06330c487ba54a8a36b1e31b54.run_cpu_time_us) (tablet-download-pool-a973de06330c487ba54a8a36b1e31b54.run_wall_time_us, tablet-download-pool-a973de06330c487ba54a8a36b1e31b54.run_wall_time_us) (tablet-download-pool-ae6d597fbc9242539edee8685a015d99.queue_time_us, tablet-download-pool-ae6d597fbc9242539edee8685a015d99.queue_time_us) (tablet-download-pool-ae6d597fbc9242539edee8685a015d99.run_cpu_time_us, tablet-download-pool-ae6d597fbc9242539edee8685a015d99.run_cpu_time_us) (tablet-download-pool-ae6d597fbc9242539edee8685a015d99.run_wall_time_us, tablet-download-pool-ae6d597fbc9242539edee8685a015d99.run_wall_time_us) (tablet-download-pool-bbae35bae8c047a6982e128ffb77ca86.queue_time_us, tablet-download-pool-bbae35bae8c047a6982e128ffb77ca86.queue_time_us) (tablet-download-pool-bbae35bae8c047a6982e128ffb77ca86.run_cpu_time_us, tablet-download-pool-bbae35bae8c047a6982e128ffb77ca86.run_cpu_time_us) (tablet-download-pool-bbae35bae8c047a6982e128ffb77ca86.run_wall_time_us, tablet-download-pool-bbae35bae8c047a6982e128ffb77ca86.run_wall_time_us) (tablet-download-pool-e61fe3cb61c240f9b2c898aff1951144.queue_time_us, tablet-download-pool-e61fe3cb61c240f9b2c898aff1951144.queue_time_us) (tablet-download-pool-e61fe3cb61c240f9b2c898aff1951144.run_cpu_time_us, tablet-download-pool-e61fe3cb61c240f9b2c898aff1951144.run_cpu_time_us) (tablet-download-pool-e61fe3cb61c240f9b2c898aff1951144.run_wall_time_us, tablet-download-pool-e61fe3cb61c240f9b2c898aff1951144.run_wall_time_us) (tablet-download-pool-f588a3ec49a24b1a9a9e19cdf8c30ca5.queue_time_us, tablet-download-pool-f588a3ec49a24b1a9a9e19cdf8c30ca5.queue_time_us) (tablet-download-pool-f588a3ec49a24b1a9a9e19cdf8c30ca5.run_cpu_time_us, tablet-download-pool-f588a3ec49a24b1a9a9e19cdf8c30ca5.run_cpu_time_us) (tablet-download-pool-f588a3ec49a24b1a9a9e19cdf8c30ca5.run_wall_time_us, tablet-download-pool-f588a3ec49a24b1a9a9e19cdf8c30ca5.run_wall_time_us) (tablet-download-pool-fb0f04b72dc046c0afee40b895e17a1d.queue_time_us, tablet-download-pool-fb0f04b72dc046c0afee40b895e17a1d.queue_time_us) (tablet-download-pool-fb0f04b72dc046c0afee40b895e17a1d.run_cpu_time_us, tablet-download-pool-fb0f04b72dc046c0afee40b895e17a1d.run_cpu_time_us) (tablet-download-pool-fb0f04b72dc046c0afee40b895e17a1d.run_wall_time_us, tablet-download-pool-fb0f04b72dc046c0afee40b895e17a1d.run_wall_time_us) (tablet-open.queue_time_us, tablet-open.queue_time_us) (tablet-open.run_cpu_time_us, tablet-open.run_cpu_time_us) (tablet-open.run_wall_time_us, tablet-open.run_wall_time_us) (txn-commit.queue_time_us, txn-commit.queue_time_us) (txn-commit.run_cpu_time_us, txn-commit.run_cpu_time_us) (txn-commit.run_wall_time_us, txn-commit.run_wall_time_us) (txn-participant-registration.queue_time_us, txn-participant-registration.queue_time_us) (txn-participant-registration.run_cpu_time_us, txn-participant-registration.run_cpu_time_us) (txn-participant-registration.run_wall_time_us, txn-participant-registration.run_wall_time_us) (txn-status-manager.queue_time_us, txn-status-manager.queue_time_us) (txn-status-manager.run_cpu_time_us, txn-status-manager.run_cpu_time_us) (txn-status-manager.run_wall_time_us, txn-status-manager.run_wall_time_us) (txn-status-tablet-reload.queue_time_us, txn-status-tablet-reload.queue_time_us) (txn-status-tablet-reload.run_cpu_time_us, txn-status-tablet-reload.run_cpu_time_us) (txn-status-tablet-reload.run_wall_time_us, txn-status-tablet-reload.run_wall_time_us) (wal-append.queue_time_us, wal-append.queue_time_us) (wal-append.run_cpu_time_us, wal-append.run_cpu_time_us) (wal-append.run_wall_time_us, wal-append.run_wall_time_us)
*** Check failure stack trace: ***
@ 0x7f1977212dcd google::LogMessage::Fail() at ??:0
@ 0x7f1977216b93 google::LogMessage::SendToLog() at ??:0
@ 0x7f19772127cc google::LogMessage::Flush() at ??:0
@ 0x7f1977213f59 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f197867c044 kudu::TraceMetrics::InternName() at ??:0
@ 0x7f1978660f98 kudu::ThreadPool::ThreadPool() at ??:0
@ 0x7f197865eefc kudu::ThreadPoolBuilder::Build() at ??:0
@ 0x7f197b21177a kudu::tserver::TabletCopyClient::TabletCopyClient() at ??:0
@ 0x7f197b21aa45 kudu::tserver::RemoteTabletCopyClient::RemoteTabletCopyClient() at ??:0
@ 0x7f197b29b76f kudu::tserver::TSTabletManager::RunTabletCopy() at ??:0
@ 0x7f197b2af4f6 kudu::tserver::TabletCopyRunnable::Run() at ??:0
@ 0x7f197b29a226 _ZZN4kudu7tserver15TSTabletManager15StartTabletCopyEPKNS_9consensus24StartTabletCopyRequestPBESt8functionIFvRKNS_6StatusENS0_24TabletServerErrorPB_CodeEEEENKUlvE_clEv at ??:0
@ 0x7f197b2a9c0a _ZNSt17_Function_handlerIFvvEZN4kudu7tserver15TSTabletManager15StartTabletCopyEPKNS1_9consensus24StartTabletCopyRequestPBESt8functionIFvRKNS1_6StatusENS2_24TabletServerErrorPB_CodeEEEEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x5630d232575c std::function<>::operator()() at ??:0
@ 0x7f1978663e06 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f19786646ef _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f197866603b _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x5630d232575c std::function<>::operator()() at ??:0
@ 0x7f1978653807 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f1978e916db start_thread at ??:0
@ 0x7f19763c971f clone at ??:0
W20240502 08:00:01.187005 20210 connection.cc:535] client connection to 127.3.139.74:45559 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 08:00:01.187183 20210 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 08:00:01.188186 20078 connection.cc:535] client connection to 127.3.139.74:45559 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 08:00:01.188266 20870 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.139.74:45559 (error 108)
W20240502 08:00:01.188292 20474 connection.cc:535] client connection to 127.3.139.74:45559 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 08:00:01.188313 20078 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 08:00:01.188390 20474 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 08:00:01.188446 20342 connection.cc:535] client connection to 127.3.139.74:45559 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 08:00:01.188576 20342 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 08:00:01.189329 20210 consensus_peers.cc:485] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 08:00:01.189867 20738 connection.cc:535] client connection to 127.3.139.74:45559 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 08:00:01.189975 20738 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 08:00:01.190297 20606 connection.cc:535] client connection to 127.3.139.74:45559 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 08:00:01.190405 20606 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 08:00:01.192505 20078 consensus_peers.cc:485] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 08:00:01.193212 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:01.193341 20738 consensus_peers.cc:485] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 08:00:01.193408 20738 consensus_peers.cc:485] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 08:00:01.193713 20606 consensus_peers.cc:485] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 08:00:01.194339 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:01.194540 20474 consensus_peers.cc:485] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 08:00:01.194603 20474 consensus_peers.cc:485] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 08:00:01.220353 20342 consensus_peers.cc:485] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 08:00:01.254724 20870 consensus_peers.cc:485] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 08:00:01.342696 20210 connection.cc:535] client connection to 127.3.139.72:43861 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 08:00:01.343245 20606 connection.cc:535] client connection to 127.3.139.72:43861 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 08:00:01.343681 20474 connection.cc:535] client connection to 127.3.139.72:43861 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 08:00:01.344480 20474 consensus_peers.cc:485] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 08:00:01.344522 26639 negotiation.cc:337] Failed RPC negotiation. Trace:
0502 08:00:01.343226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.139.72:43861 (local address 127.3.139.71:48783)
0502 08:00:01.343415 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0502 08:00:01.343426 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0502 08:00:01.343518 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0502 08:00:01.343943 (+ 425us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":89}
W20240502 08:00:01.344519 26631 negotiation.cc:337] Failed RPC negotiation. Trace:
0502 08:00:01.343250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.139.72:43861 (local address 127.3.139.66:56103)
0502 08:00:01.343399 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0502 08:00:01.343414 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0502 08:00:01.343518 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0502 08:00:01.344184 (+ 666us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":82}
W20240502 08:00:01.345021 20210 consensus_peers.cc:485] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 1: this message will repeat every 5th retry.
W20240502 08:00:01.345168 20738 connection.cc:535] client connection to 127.3.139.72:43861 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 08:00:01.345556 20606 consensus_peers.cc:485] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 08:00:01.345618 20606 consensus_peers.cc:485] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 08:00:01.345953 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 08:00:01.346081 20738 consensus_peers.cc:485] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 08:00:01.350122 20870 consensus_peers.cc:485] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 08:00:01.364564 20606 consensus_peers.cc:485] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 08:00:01.375450 20078 consensus_peers.cc:485] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 08:00:01.419790 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:01.421036 20606 consensus_peers.cc:485] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 08:00:01.468297 20606 consensus_peers.cc:485] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 08:00:01.494772 20870 consensus_peers.cc:485] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 08:00:01.509989 20210 consensus_peers.cc:485] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 08:00:01.552678 20342 consensus_peers.cc:485] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 08:00:01.569214 20342 consensus_peers.cc:485] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 08:00:01.590771 20078 consensus_peers.cc:485] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 08:00:01.602104 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:01.648514 20870 consensus_peers.cc:485] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 08:00:01.713806 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:01.956748 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:01.988741 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:02.190449 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:02.377562 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:02.440843 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:02.578655 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:02.821570 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:02.849997 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:03.193217 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:03.241322 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:03.357928 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:03.361840 20078 consensus_peers.cc:485] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 08:00:03.471475 20474 consensus_peers.cc:485] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 08:00:03.495913 20738 consensus_peers.cc:485] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 08:00:03.513062 20474 consensus_peers.cc:485] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 08:00:03.519611 20210 consensus_peers.cc:485] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 08:00:03.566704 20606 consensus_peers.cc:485] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 08:00:03.582736 20606 consensus_peers.cc:485] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 08:00:03.585865 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:03.593585 20738 consensus_peers.cc:485] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 08:00:03.639317 20210 consensus_peers.cc:485] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 08:00:03.652928 20474 consensus_peers.cc:485] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 08:00:03.658375 20738 consensus_peers.cc:485] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 08:00:03.705068 20606 consensus_peers.cc:485] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 08:00:03.740329 20870 consensus_peers.cc:485] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 08:00:03.766470 20342 consensus_peers.cc:485] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 08:00:03.788544 20342 consensus_peers.cc:485] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 08:00:03.797439 20078 consensus_peers.cc:485] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 08:00:03.834239 20606 consensus_peers.cc:485] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 08:00:03.858036 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:03.865428 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:03.884614 20210 consensus_peers.cc:485] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 08:00:03.886955 20870 consensus_peers.cc:485] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 08:00:03.896739 20342 consensus_peers.cc:485] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 08:00:03.974653 20606 consensus_peers.cc:485] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 08:00:04.015278 20870 consensus_peers.cc:485] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 08:00:04.137887 20606 consensus_peers.cc:485] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 08:00:04.143535 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:04.170545 20078 consensus_peers.cc:485] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 08:00:04.240401 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:04.253608 20870 consensus_peers.cc:485] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 08:00:04.429013 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:04.702351 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:04.814710 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:05.001709 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:05.212980 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:05.262406 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:05.477875 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:05.684469 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:05.724941 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:05.734058 20078 consensus_peers.cc:485] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 08:00:05.867491 20474 consensus_peers.cc:485] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 08:00:05.869299 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:05.885089 20210 consensus_peers.cc:485] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 08:00:06.088210 20870 consensus_peers.cc:485] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 08:00:06.107362 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:06.119539 20342 consensus_peers.cc:485] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 08:00:06.122251 20738 consensus_peers.cc:485] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 08:00:06.183207 20738 consensus_peers.cc:485] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 08:00:06.199656 20870 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111) [suppressed 60 similar messages]
W20240502 08:00:06.200208 20474 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111) [suppressed 30 similar messages]
W20240502 08:00:06.200616 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:06.201228 20474 consensus_peers.cc:485] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 08:00:06.235275 20078 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111) [suppressed 29 similar messages]
W20240502 08:00:06.264102 20342 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111) [suppressed 42 similar messages]
W20240502 08:00:06.264719 20342 consensus_peers.cc:485] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 08:00:06.293735 20738 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111) [suppressed 31 similar messages]
W20240502 08:00:06.294391 20738 consensus_peers.cc:485] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 08:00:06.301203 20606 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111) [suppressed 59 similar messages]
W20240502 08:00:06.301797 20606 consensus_peers.cc:485] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 08:00:06.311256 20210 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111) [suppressed 30 similar messages]
W20240502 08:00:06.311949 20210 consensus_peers.cc:485] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 08:00:06.340433 20210 consensus_peers.cc:485] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 08:00:06.346799 20078 consensus_peers.cc:485] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 08:00:06.346982 20606 consensus_peers.cc:485] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 08:00:06.365954 20606 consensus_peers.cc:485] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 08:00:06.387639 20870 consensus_peers.cc:485] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 08:00:06.396037 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:06.403592 20606 consensus_peers.cc:485] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 08:00:06.434573 20474 consensus_peers.cc:485] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 08:00:06.438365 20870 consensus_peers.cc:485] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 08:00:06.440169 20606 consensus_peers.cc:485] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 08:00:06.466225 20342 consensus_peers.cc:485] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 08:00:06.575683 20606 consensus_peers.cc:485] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 08:00:06.582955 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:06.662878 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:06.728649 20078 consensus_peers.cc:485] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 08:00:06.811776 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:07.009770 20870 consensus_peers.cc:485] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 08:00:07.041690 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:07.186856 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:07.220469 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:07.567651 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:07.705477 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:07.768880 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:08.024056 20078 consensus_peers.cc:485] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20240502 08:00:08.161654 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:08.181702 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:08.203672 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:08.273312 20474 consensus_peers.cc:485] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20240502 08:00:08.378749 20210 consensus_peers.cc:485] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20240502 08:00:08.465895 20738 consensus_peers.cc:485] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20240502 08:00:08.510711 20870 consensus_peers.cc:485] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20240502 08:00:08.585616 20870 consensus_peers.cc:485] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20240502 08:00:08.592729 20342 consensus_peers.cc:485] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20240502 08:00:08.617933 20342 consensus_peers.cc:485] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20240502 08:00:08.633998 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:08.639719 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:08.688097 20738 consensus_peers.cc:485] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20240502 08:00:08.696889 20606 consensus_peers.cc:485] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20240502 08:00:08.744544 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:08.773038 20474 consensus_peers.cc:485] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20240502 08:00:08.794958 20078 consensus_peers.cc:485] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20240502 08:00:08.803010 20210 consensus_peers.cc:485] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20240502 08:00:08.822361 20738 consensus_peers.cc:485] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20240502 08:00:08.829396 20606 consensus_peers.cc:485] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20240502 08:00:08.834718 20474 consensus_peers.cc:485] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20240502 08:00:08.910249 20606 consensus_peers.cc:485] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20240502 08:00:08.926492 20210 consensus_peers.cc:485] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20240502 08:00:08.962327 20870 consensus_peers.cc:485] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20240502 08:00:08.963524 20606 consensus_peers.cc:485] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20240502 08:00:08.999716 20342 consensus_peers.cc:485] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20240502 08:00:09.036242 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:09.068009 20606 consensus_peers.cc:485] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20240502 08:00:09.126659 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:09.162854 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:09.209441 20078 consensus_peers.cc:485] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20240502 08:00:09.235349 20606 consensus_peers.cc:485] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20240502 08:00:09.503965 20870 consensus_peers.cc:485] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20240502 08:00:09.523510 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:09.570017 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:09.635910 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:10.017867 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:10.051409 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:10.146870 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:10.432703 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:10.528932 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:10.557345 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:10.752830 20078 consensus_peers.cc:485] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20240502 08:00:10.773128 20870 consensus_peers.cc:485] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20240502 08:00:10.890754 20474 consensus_peers.cc:485] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20240502 08:00:10.972069 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:10.977813 20210 consensus_peers.cc:485] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20240502 08:00:11.035394 20738 consensus_peers.cc:485] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20240502 08:00:11.068060 20342 consensus_peers.cc:485] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20240502 08:00:11.068491 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:11.069380 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:11.165016 20870 consensus_peers.cc:485] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20240502 08:00:11.171233 20342 consensus_peers.cc:485] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20240502 08:00:11.182181 20870 consensus_peers.cc:485] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20240502 08:00:11.224061 20606 consensus_peers.cc:485] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20240502 08:00:11.228622 20078 consensus_peers.cc:485] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20240502 08:00:11.230832 20474 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111) [suppressed 29 similar messages]
W20240502 08:00:11.231467 20474 consensus_peers.cc:485] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20240502 08:00:11.240528 20078 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111) [suppressed 31 similar messages]
W20240502 08:00:11.251488 20738 consensus_peers.cc:485] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20240502 08:00:11.322643 20606 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111) [suppressed 60 similar messages]
W20240502 08:00:11.323269 20606 consensus_peers.cc:485] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20240502 08:00:11.335201 20606 consensus_peers.cc:485] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20240502 08:00:11.338340 20870 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111) [suppressed 63 similar messages]
W20240502 08:00:11.384507 20738 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111) [suppressed 29 similar messages]
W20240502 08:00:11.385004 20738 consensus_peers.cc:485] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20240502 08:00:11.444469 20210 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111) [suppressed 29 similar messages]
W20240502 08:00:11.444994 20210 consensus_peers.cc:485] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20240502 08:00:11.448568 20474 consensus_peers.cc:485] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20240502 08:00:11.492925 20606 consensus_peers.cc:485] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20240502 08:00:11.499795 20210 consensus_peers.cc:485] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20240502 08:00:11.506098 20342 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111) [suppressed 40 similar messages]
W20240502 08:00:11.506701 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:11.515663 20342 consensus_peers.cc:485] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20240502 08:00:11.543356 20606 consensus_peers.cc:485] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20240502 08:00:11.567507 20606 consensus_peers.cc:485] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20240502 08:00:11.584862 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:11.624976 20078 consensus_peers.cc:485] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20240502 08:00:11.692101 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:11.943660 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:11.967516 20870 consensus_peers.cc:485] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20240502 08:00:12.154628 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:12.202993 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:12.361248 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:12.574939 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:12.731298 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:12.807742 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:13.124954 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:13.242467 20078 consensus_peers.cc:485] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20240502 08:00:13.265247 20870 consensus_peers.cc:485] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20240502 08:00:13.270705 20078 consensus_peers.cc:485] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20240502 08:00:13.303791 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:13.306327 20474 consensus_peers.cc:485] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20240502 08:00:13.335742 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:13.354765 20474 consensus_peers.cc:485] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20240502 08:00:13.432232 20342 consensus_peers.cc:485] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20240502 08:00:13.467104 20606 consensus_peers.cc:485] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20240502 08:00:13.568575 20210 consensus_peers.cc:485] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20240502 08:00:13.586948 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:13.593194 20738 consensus_peers.cc:485] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20240502 08:00:13.597813 20870 consensus_peers.cc:485] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20240502 08:00:13.630275 20738 consensus_peers.cc:485] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20240502 08:00:13.638450 20870 consensus_peers.cc:485] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20240502 08:00:13.712589 20606 consensus_peers.cc:485] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20240502 08:00:13.743467 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:13.746735 20606 consensus_peers.cc:485] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20240502 08:00:13.907670 20342 consensus_peers.cc:485] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20240502 08:00:13.912047 20210 consensus_peers.cc:485] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20240502 08:00:13.951501 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:13.955027 20606 consensus_peers.cc:485] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20240502 08:00:13.983141 20738 consensus_peers.cc:485] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20240502 08:00:13.995875 20606 consensus_peers.cc:485] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20240502 08:00:14.010507 20078 consensus_peers.cc:485] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20240502 08:00:14.032351 20606 consensus_peers.cc:485] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20240502 08:00:14.078970 20474 consensus_peers.cc:485] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20240502 08:00:14.086366 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:14.094832 20342 consensus_peers.cc:485] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20240502 08:00:14.180431 20210 consensus_peers.cc:485] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20240502 08:00:14.314204 20870 consensus_peers.cc:485] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20240502 08:00:14.365587 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:14.366868 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:14.598378 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:14.778121 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:14.820200 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:14.987007 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:15.171797 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:15.325917 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:15.406657 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:15.687980 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:15.691674 20342 consensus_peers.cc:485] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20240502 08:00:15.752934 20474 consensus_peers.cc:485] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20240502 08:00:15.792783 20078 consensus_peers.cc:485] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20240502 08:00:15.847440 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:15.850159 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:15.853413 20078 consensus_peers.cc:485] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20240502 08:00:15.884325 20474 consensus_peers.cc:485] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20240502 08:00:15.912621 20606 consensus_peers.cc:485] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20240502 08:00:15.976552 20606 consensus_peers.cc:485] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20240502 08:00:16.010071 20738 consensus_peers.cc:485] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20240502 08:00:16.013417 20738 consensus_peers.cc:485] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20240502 08:00:16.042436 20870 consensus_peers.cc:485] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20240502 08:00:16.082886 20870 consensus_peers.cc:485] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20240502 08:00:16.197332 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:16.201463 20870 consensus_peers.cc:485] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20240502 08:00:16.230942 20210 consensus_peers.cc:485] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20240502 08:00:16.247174 20078 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111) [suppressed 29 similar messages]
W20240502 08:00:16.274348 20606 consensus_peers.cc:485] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20240502 08:00:16.303560 20474 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111) [suppressed 29 similar messages]
W20240502 08:00:16.324100 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:16.363261 20870 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111) [suppressed 60 similar messages]
W20240502 08:00:16.363821 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:16.416289 20606 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111) [suppressed 61 similar messages]
W20240502 08:00:16.416882 20606 consensus_peers.cc:485] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20240502 08:00:16.430672 20078 consensus_peers.cc:485] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20240502 08:00:16.460556 20606 consensus_peers.cc:485] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20240502 08:00:16.464241 20738 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111) [suppressed 29 similar messages]
W20240502 08:00:16.464771 20738 consensus_peers.cc:485] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20240502 08:00:16.473485 20210 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111) [suppressed 29 similar messages]
W20240502 08:00:16.474056 20210 consensus_peers.cc:485] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20240502 08:00:16.525540 20606 consensus_peers.cc:485] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20240502 08:00:16.594194 20342 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111) [suppressed 40 similar messages]
W20240502 08:00:16.594882 20342 consensus_peers.cc:485] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20240502 08:00:16.659111 20210 consensus_peers.cc:485] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20240502 08:00:16.696786 20870 consensus_peers.cc:485] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20240502 08:00:16.710922 20474 consensus_peers.cc:485] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20240502 08:00:16.759346 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:16.766644 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:16.794207 20342 consensus_peers.cc:485] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20240502 08:00:16.904359 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:17.245061 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:17.364976 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:17.379313 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:17.672577 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:17.911348 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:17.957234 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:18.112942 20342 consensus_peers.cc:485] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20240502 08:00:18.127157 20078 consensus_peers.cc:485] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20240502 08:00:18.151335 20078 consensus_peers.cc:485] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20240502 08:00:18.199859 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:18.205091 20474 consensus_peers.cc:485] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20240502 08:00:18.356354 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:18.403666 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:18.414196 20606 consensus_peers.cc:485] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20240502 08:00:18.437683 20870 consensus_peers.cc:485] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20240502 08:00:18.440806 20738 consensus_peers.cc:485] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20240502 08:00:18.469797 20474 consensus_peers.cc:485] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20240502 08:00:18.476316 20606 consensus_peers.cc:485] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20240502 08:00:18.542171 20870 consensus_peers.cc:485] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20240502 08:00:18.573649 20738 consensus_peers.cc:485] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20240502 08:00:18.633947 20870 consensus_peers.cc:485] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20240502 08:00:18.652679 20210 consensus_peers.cc:485] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20240502 08:00:18.709956 20078 consensus_peers.cc:485] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20240502 08:00:18.731922 20606 consensus_peers.cc:485] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20240502 08:00:18.742002 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:18.887271 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:18.935158 20606 consensus_peers.cc:485] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20240502 08:00:18.960160 20342 consensus_peers.cc:485] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20240502 08:00:18.980331 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:19.006731 20606 consensus_peers.cc:485] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20240502 08:00:19.006814 20210 consensus_peers.cc:485] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20240502 08:00:19.166240 20210 consensus_peers.cc:485] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20240502 08:00:19.170413 20870 consensus_peers.cc:485] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20240502 08:00:19.178279 20738 consensus_peers.cc:485] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20240502 08:00:19.178884 20606 consensus_peers.cc:485] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20240502 08:00:19.210613 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:19.293413 20474 consensus_peers.cc:485] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20240502 08:00:19.336383 20342 consensus_peers.cc:485] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20240502 08:00:19.362397 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:19.449031 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:19.752159 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:19.807544 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:19.896569 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:20.197579 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:20.322125 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:20.426487 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:20.445657 20342 consensus_peers.cc:485] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20240502 08:00:20.602083 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:20.692013 20606 consensus_peers.cc:485] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20240502 08:00:20.695799 20078 consensus_peers.cc:485] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20240502 08:00:20.788846 20210 consensus_peers.cc:485] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20240502 08:00:20.802191 20474 consensus_peers.cc:485] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20240502 08:00:20.808821 20474 consensus_peers.cc:485] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20240502 08:00:20.820484 20606 consensus_peers.cc:485] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20240502 08:00:20.846894 20870 consensus_peers.cc:485] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20240502 08:00:20.882725 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:20.916946 20870 consensus_peers.cc:485] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20240502 08:00:20.958009 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:21.005331 20078 consensus_peers.cc:485] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20240502 08:00:21.054944 20606 consensus_peers.cc:485] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20240502 08:00:21.072921 20738 consensus_peers.cc:485] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20240502 08:00:21.154958 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:21.156821 20078 consensus_peers.cc:485] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20240502 08:00:21.256312 20870 consensus_peers.cc:485] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20240502 08:00:21.273037 20078 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111) [suppressed 30 similar messages]
W20240502 08:00:21.312029 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:21.327100 20474 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111) [suppressed 30 similar messages]
W20240502 08:00:21.372717 20738 consensus_peers.cc:485] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20240502 08:00:21.377394 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:21.386888 20870 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111) [suppressed 61 similar messages]
W20240502 08:00:21.432049 20210 consensus_peers.cc:485] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20240502 08:00:21.517356 20606 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111) [suppressed 61 similar messages]
W20240502 08:00:21.517905 20606 consensus_peers.cc:485] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20240502 08:00:21.633415 20474 consensus_peers.cc:485] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20240502 08:00:21.635303 20342 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111) [suppressed 40 similar messages]
W20240502 08:00:21.635865 20342 consensus_peers.cc:485] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20240502 08:00:21.672811 20738 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111) [suppressed 29 similar messages]
W20240502 08:00:21.680526 20210 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111) [suppressed 31 similar messages]
W20240502 08:00:21.688506 20606 consensus_peers.cc:485] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20240502 08:00:21.737124 20738 consensus_peers.cc:485] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20240502 08:00:21.744786 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:21.752769 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:21.880791 20606 consensus_peers.cc:485] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20240502 08:00:21.961961 20870 consensus_peers.cc:485] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20240502 08:00:21.968010 20210 consensus_peers.cc:485] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20240502 08:00:21.996243 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:22.009635 20342 consensus_peers.cc:485] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20240502 08:00:22.162606 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:22.300932 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:22.494206 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:22.767998 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:22.830891 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:22.893774 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:22.975998 20342 consensus_peers.cc:485] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20240502 08:00:23.211331 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:23.223470 20870 consensus_peers.cc:485] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20240502 08:00:23.234030 20474 consensus_peers.cc:485] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20240502 08:00:23.268131 20210 consensus_peers.cc:485] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20240502 08:00:23.275061 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:23.280817 20870 consensus_peers.cc:485] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20240502 08:00:23.292477 20606 consensus_peers.cc:485] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20240502 08:00:23.322252 20606 consensus_peers.cc:485] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20240502 08:00:23.370648 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:23.384433 20474 consensus_peers.cc:485] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20240502 08:00:23.427583 20078 consensus_peers.cc:485] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20240502 08:00:23.620126 20078 consensus_peers.cc:485] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20240502 08:00:23.645562 20210 consensus_peers.cc:485] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20240502 08:00:23.705248 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:23.763358 20738 consensus_peers.cc:485] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20240502 08:00:23.790652 20078 consensus_peers.cc:485] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20240502 08:00:23.840201 20606 consensus_peers.cc:485] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20240502 08:00:23.840806 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:23.856973 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:23.879935 20870 consensus_peers.cc:485] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20240502 08:00:23.965767 20474 consensus_peers.cc:485] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20240502 08:00:24.018085 20606 consensus_peers.cc:485] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20240502 08:00:24.053099 20738 consensus_peers.cc:485] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20240502 08:00:24.081511 20738 consensus_peers.cc:485] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20240502 08:00:24.125625 20606 consensus_peers.cc:485] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20240502 08:00:24.161686 20342 consensus_peers.cc:485] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20240502 08:00:24.220299 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:24.275394 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:24.363988 20210 consensus_peers.cc:485] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20240502 08:00:24.393638 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:24.412441 20606 consensus_peers.cc:485] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20240502 08:00:24.430119 20870 consensus_peers.cc:485] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20240502 08:00:24.525058 20342 consensus_peers.cc:485] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20240502 08:00:24.696151 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:24.785985 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:24.863305 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:25.108723 20342 consensus_peers.cc:485] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20240502 08:00:25.259800 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:25.289700 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:25.386279 20870 consensus_peers.cc:485] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20240502 08:00:25.458596 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:25.585894 20474 consensus_peers.cc:485] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20240502 08:00:25.675596 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:25.690909 20870 consensus_peers.cc:485] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20240502 08:00:25.776553 20210 consensus_peers.cc:485] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20240502 08:00:25.804332 20474 consensus_peers.cc:485] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20240502 08:00:25.815668 20606 consensus_peers.cc:485] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20240502 08:00:25.820565 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:25.869928 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:25.989635 20606 consensus_peers.cc:485] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20240502 08:00:26.087108 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:26.107484 20078 consensus_peers.cc:485] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20240502 08:00:26.128679 20078 consensus_peers.cc:485] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20240502 08:00:26.158259 20870 consensus_peers.cc:485] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20240502 08:00:26.217530 20210 consensus_peers.cc:485] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20240502 08:00:26.260818 20738 consensus_peers.cc:485] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20240502 08:00:26.269671 20078 consensus_peers.cc:485] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20240502 08:00:26.274526 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:26.340097 20606 consensus_peers.cc:485] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20240502 08:00:26.397231 20474 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111) [suppressed 31 similar messages]
W20240502 08:00:26.397830 20474 consensus_peers.cc:485] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20240502 08:00:26.411916 20870 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111) [suppressed 59 similar messages]
W20240502 08:00:26.412473 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:26.503641 20078 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111) [suppressed 29 similar messages]
W20240502 08:00:26.508072 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:26.528180 20738 consensus_peers.cc:485] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20240502 08:00:26.534838 20606 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111) [suppressed 59 similar messages]
W20240502 08:00:26.535405 20606 consensus_peers.cc:485] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20240502 08:00:26.556126 20738 consensus_peers.cc:485] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20240502 08:00:26.651482 20342 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111) [suppressed 40 similar messages]
W20240502 08:00:26.652077 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:26.659691 20606 consensus_peers.cc:485] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20240502 08:00:26.694533 20738 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111) [suppressed 30 similar messages]
W20240502 08:00:26.710745 20210 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111) [suppressed 30 similar messages]
W20240502 08:00:26.749027 20342 consensus_peers.cc:485] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20240502 08:00:26.765893 20210 consensus_peers.cc:485] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20240502 08:00:26.782706 20606 consensus_peers.cc:485] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20240502 08:00:26.851250 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:26.928208 20342 consensus_peers.cc:485] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20240502 08:00:27.033687 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:27.107604 20870 consensus_peers.cc:485] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20240502 08:00:27.191758 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:27.377826 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:27.381219 20342 consensus_peers.cc:485] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20240502 08:00:27.600080 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:27.609015 20870 consensus_peers.cc:485] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20240502 08:00:27.744269 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:27.964586 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:28.093205 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:28.094923 20474 consensus_peers.cc:485] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20240502 08:00:28.097105 20210 consensus_peers.cc:485] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20240502 08:00:28.166376 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:28.236667 20606 consensus_peers.cc:485] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20240502 08:00:28.283324 20606 consensus_peers.cc:485] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20240502 08:00:28.284094 20870 consensus_peers.cc:485] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20240502 08:00:28.294435 20474 consensus_peers.cc:485] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20240502 08:00:28.378338 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:28.518723 20078 consensus_peers.cc:485] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20240502 08:00:28.530174 20210 consensus_peers.cc:485] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20240502 08:00:28.574705 20078 consensus_peers.cc:485] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20240502 08:00:28.600916 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:28.609086 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:28.653472 20870 consensus_peers.cc:485] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20240502 08:00:28.676841 20078 consensus_peers.cc:485] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20240502 08:00:28.732419 20738 consensus_peers.cc:485] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20240502 08:00:28.872560 20606 consensus_peers.cc:485] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20240502 08:00:28.904368 20606 consensus_peers.cc:485] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20240502 08:00:28.952829 20606 consensus_peers.cc:485] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20240502 08:00:28.956856 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:28.980651 20738 consensus_peers.cc:485] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20240502 08:00:29.013269 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:29.023847 20210 consensus_peers.cc:485] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20240502 08:00:29.051039 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:29.066005 20474 consensus_peers.cc:485] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20240502 08:00:29.170542 20738 consensus_peers.cc:485] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20240502 08:00:29.328517 20342 consensus_peers.cc:485] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20240502 08:00:29.410852 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:29.423339 20606 consensus_peers.cc:485] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20240502 08:00:29.559986 20342 consensus_peers.cc:485] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20240502 08:00:29.569005 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:29.592969 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:29.666391 20870 consensus_peers.cc:485] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20240502 08:00:29.873998 20342 consensus_peers.cc:485] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20240502 08:00:29.905385 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:30.041954 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:30.060372 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:30.131656 20870 consensus_peers.cc:485] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20240502 08:00:30.354364 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:30.451072 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:30.509647 20870 consensus_peers.cc:485] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20240502 08:00:30.526957 20474 consensus_peers.cc:485] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20240502 08:00:30.544045 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:30.799870 20210 consensus_peers.cc:485] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20240502 08:00:30.806536 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:30.894018 20606 consensus_peers.cc:485] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20240502 08:00:30.894554 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:30.931391 20474 consensus_peers.cc:485] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20240502 08:00:30.958981 20606 consensus_peers.cc:485] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20240502 08:00:31.021059 20078 consensus_peers.cc:485] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20240502 08:00:31.041571 20078 consensus_peers.cc:485] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20240502 08:00:31.092504 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:31.157766 20210 consensus_peers.cc:485] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20240502 08:00:31.181587 20870 consensus_peers.cc:485] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20240502 08:00:31.226229 20078 consensus_peers.cc:485] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20240502 08:00:31.309646 20738 consensus_peers.cc:485] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20240502 08:00:31.315151 20606 consensus_peers.cc:485] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20240502 08:00:31.323613 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:31.360610 20606 consensus_peers.cc:485] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20240502 08:00:31.405141 20738 consensus_peers.cc:485] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20240502 08:00:31.432848 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:31.436326 20870 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111) [suppressed 60 similar messages]
W20240502 08:00:31.463137 20474 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111) [suppressed 28 similar messages]
W20240502 08:00:31.531878 20078 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111) [suppressed 29 similar messages]
W20240502 08:00:31.541891 20606 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111) [suppressed 59 similar messages]
W20240502 08:00:31.542690 20606 consensus_peers.cc:485] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20240502 08:00:31.643357 20474 consensus_peers.cc:485] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20240502 08:00:31.644055 20210 consensus_peers.cc:485] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20240502 08:00:31.697024 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:31.774510 20738 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111) [suppressed 28 similar messages]
W20240502 08:00:31.787338 20210 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111) [suppressed 30 similar messages]
W20240502 08:00:31.810529 20738 consensus_peers.cc:485] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20240502 08:00:31.833035 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:31.871726 20342 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111) [suppressed 40 similar messages]
W20240502 08:00:31.872501 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:31.964015 20342 consensus_peers.cc:485] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20240502 08:00:31.979233 20606 consensus_peers.cc:485] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20240502 08:00:32.004837 20342 consensus_peers.cc:485] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20240502 08:00:32.047488 20870 consensus_peers.cc:485] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20240502 08:00:32.110960 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:32.330214 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:32.335839 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:32.431145 20342 consensus_peers.cc:485] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20240502 08:00:32.491182 20870 consensus_peers.cc:485] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20240502 08:00:32.673462 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:32.738216 20474 consensus_peers.cc:485] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20240502 08:00:32.749882 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:32.943666 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:33.082194 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:33.249508 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:33.295970 20870 consensus_peers.cc:485] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20240502 08:00:33.390416 20210 consensus_peers.cc:485] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20240502 08:00:33.473472 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:33.489610 20606 consensus_peers.cc:485] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20240502 08:00:33.542639 20078 consensus_peers.cc:485] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20240502 08:00:33.546911 20078 consensus_peers.cc:485] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20240502 08:00:33.555976 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:33.602854 20210 consensus_peers.cc:485] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20240502 08:00:33.638340 20606 consensus_peers.cc:485] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20240502 08:00:33.659718 20738 consensus_peers.cc:485] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20240502 08:00:33.665714 20474 consensus_peers.cc:485] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20240502 08:00:33.778797 20606 consensus_peers.cc:485] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20240502 08:00:33.783565 20606 consensus_peers.cc:485] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20240502 08:00:33.795846 20738 consensus_peers.cc:485] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20240502 08:00:33.817520 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:33.882841 20870 consensus_peers.cc:485] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20240502 08:00:33.951946 20078 consensus_peers.cc:485] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20240502 08:00:33.962883 20606 consensus_peers.cc:485] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20240502 08:00:33.970957 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:34.076902 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:34.184523 20342 consensus_peers.cc:485] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20240502 08:00:34.281966 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:34.323711 20606 consensus_peers.cc:485] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20240502 08:00:34.328212 20738 consensus_peers.cc:485] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20240502 08:00:34.366052 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:34.479012 20474 consensus_peers.cc:485] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20240502 08:00:34.519858 20210 consensus_peers.cc:485] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20240502 08:00:34.637926 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:34.675700 20342 consensus_peers.cc:485] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20240502 08:00:34.784505 20870 consensus_peers.cc:485] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20240502 08:00:34.832995 20870 consensus_peers.cc:485] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20240502 08:00:34.887380 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:34.905691 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:34.947635 20342 consensus_peers.cc:485] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20240502 08:00:35.029367 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:35.334645 20474 consensus_peers.cc:485] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20240502 08:00:35.447302 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:35.464143 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:35.515304 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:35.797139 20870 consensus_peers.cc:485] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20240502 08:00:35.894349 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:35.899470 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:35.982085 20078 consensus_peers.cc:485] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20240502 08:00:36.055203 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:36.074072 20474 consensus_peers.cc:485] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20240502 08:00:36.090188 20738 consensus_peers.cc:485] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20240502 08:00:36.188961 20606 consensus_peers.cc:485] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20240502 08:00:36.225324 20210 consensus_peers.cc:485] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20240502 08:00:36.270987 20078 consensus_peers.cc:485] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20240502 08:00:36.301208 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:36.332192 20738 consensus_peers.cc:485] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20240502 08:00:36.354241 20606 consensus_peers.cc:485] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20240502 08:00:36.449398 20210 consensus_peers.cc:485] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20240502 08:00:36.467592 20606 consensus_peers.cc:485] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20240502 08:00:36.469480 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:36.482776 20474 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111) [suppressed 29 similar messages]
W20240502 08:00:36.487006 20870 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111) [suppressed 58 similar messages]
W20240502 08:00:36.487602 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:36.532419 20078 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111) [suppressed 29 similar messages]
W20240502 08:00:36.532982 20078 consensus_peers.cc:485] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20240502 08:00:36.567731 20606 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111) [suppressed 57 similar messages]
W20240502 08:00:36.574358 20606 consensus_peers.cc:485] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20240502 08:00:36.606290 20870 consensus_peers.cc:485] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20240502 08:00:36.699270 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:36.715010 20210 consensus_peers.cc:485] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20240502 08:00:36.776852 20342 consensus_peers.cc:485] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20240502 08:00:36.848181 20738 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111) [suppressed 30 similar messages]
W20240502 08:00:36.848750 20738 consensus_peers.cc:485] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20240502 08:00:36.938153 20606 consensus_peers.cc:485] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20240502 08:00:36.942283 20606 consensus_peers.cc:485] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20240502 08:00:36.956339 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:36.973121 20342 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111) [suppressed 41 similar messages]
W20240502 08:00:36.973630 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:36.985870 20342 consensus_peers.cc:485] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20240502 08:00:36.987856 20210 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111) [suppressed 28 similar messages]
W20240502 08:00:36.988579 20474 consensus_peers.cc:485] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20240502 08:00:37.025570 20870 consensus_peers.cc:485] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20240502 08:00:37.109278 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:37.245571 20342 consensus_peers.cc:485] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20240502 08:00:37.421810 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:37.457667 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:37.667623 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:37.703270 20870 consensus_peers.cc:485] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20240502 08:00:37.858492 20474 consensus_peers.cc:485] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20240502 08:00:37.938040 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:38.015856 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:38.123780 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:38.312690 20870 consensus_peers.cc:485] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20240502 08:00:38.452550 20474 consensus_peers.cc:485] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20240502 08:00:38.493008 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:38.501052 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:38.515094 20078 consensus_peers.cc:485] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20240502 08:00:38.558655 20738 consensus_peers.cc:485] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20240502 08:00:38.609290 20078 consensus_peers.cc:485] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20240502 08:00:38.660876 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:38.813336 20078 consensus_peers.cc:485] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20240502 08:00:38.819954 20606 consensus_peers.cc:485] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20240502 08:00:38.826719 20210 consensus_peers.cc:485] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20240502 08:00:38.839443 20738 consensus_peers.cc:485] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20240502 08:00:38.872249 20606 consensus_peers.cc:485] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20240502 08:00:38.873322 20870 consensus_peers.cc:485] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20240502 08:00:38.895634 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:38.971174 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:39.049474 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:39.102459 20606 consensus_peers.cc:485] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20240502 08:00:39.137338 20210 consensus_peers.cc:485] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20240502 08:00:39.257994 20342 consensus_peers.cc:485] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20240502 08:00:39.286790 20738 consensus_peers.cc:485] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20240502 08:00:39.313228 20606 consensus_peers.cc:485] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20240502 08:00:39.326983 20870 consensus_peers.cc:485] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20240502 08:00:39.415026 20606 consensus_peers.cc:485] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20240502 08:00:39.441366 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:39.452162 20210 consensus_peers.cc:485] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20240502 08:00:39.469456 20474 consensus_peers.cc:485] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20240502 08:00:39.474414 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:39.540741 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:39.672713 20606 consensus_peers.cc:485] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20240502 08:00:39.692224 20342 consensus_peers.cc:485] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20240502 08:00:39.831696 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:39.937564 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:39.965047 20342 consensus_peers.cc:485] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20240502 08:00:40.060410 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:40.242238 20870 consensus_peers.cc:485] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20240502 08:00:40.301827 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:40.313745 20474 consensus_peers.cc:485] T dbf4b90cac9e4ff3b2936a1fa2ca8561 P e951f009258e473cb57cf8773c7975d4 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20240502 08:00:40.351555 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:40.669696 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:40.726687 20474 consensus_peers.cc:485] T a973de06330c487ba54a8a36b1e31b54 P e951f009258e473cb57cf8773c7975d4 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20240502 08:00:40.752092 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:40.778892 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:40.992110 20078 consensus_peers.cc:485] T 880876d792554606a7ccf21f9ec08e0f P 29a7180fe4574c09a8dce7c27500aab2 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20240502 08:00:41.044584 20870 consensus_peers.cc:485] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20240502 08:00:41.077360 20738 consensus_peers.cc:485] T 6a77bda800d44c4bbe1b082a5ea05b55 P c15e23a34059456788f9775031e27ec0 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20240502 08:00:41.108355 20078 consensus_peers.cc:485] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20240502 08:00:41.146189 20738 consensus_peers.cc:485] T b73e0ce533014420af513429620af68c P c15e23a34059456788f9775031e27ec0 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20240502 08:00:41.204881 20606 consensus_peers.cc:485] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20240502 08:00:41.235505 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:41.333985 20210 consensus_peers.cc:485] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20240502 08:00:41.374548 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:41.375717 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:41.438226 20870 consensus_peers.cc:485] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20240502 08:00:41.484179 20078 consensus_peers.cc:485] T 9ad40762dc784dfa9f3507e025db32b8 P 29a7180fe4574c09a8dce7c27500aab2 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20240502 08:00:41.515132 20870 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111) [suppressed 60 similar messages]
W20240502 08:00:41.518741 20210 consensus_peers.cc:485] T ae6d597fbc9242539edee8685a015d99 P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20240502 08:00:41.576628 20606 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111) [suppressed 57 similar messages]
W20240502 08:00:41.577613 20606 consensus_peers.cc:485] T 0f03a9b07a8c40bdbdf3ff714302a3ea P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20240502 08:00:41.754177 20606 consensus_peers.cc:485] T c3ca6f24c92d4273ac6c9d83161813d7 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20240502 08:00:41.772624 20474 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111) [suppressed 31 similar messages]
W20240502 08:00:41.774302 20606 consensus_peers.cc:485] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20240502 08:00:41.796775 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:41.824522 20342 consensus_peers.cc:485] T 574fdd7a3c114ac9928ac12d1a7a4186 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20240502 08:00:41.866822 20210 consensus_peers.cc:485] T 0e2908de10684c84af542f16ff8727cb P b7604c0eb3bf4a7cb260b564a94a1191 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20240502 08:00:41.876209 20738 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111) [suppressed 30 similar messages]
W20240502 08:00:41.876801 20738 consensus_peers.cc:485] T 331368e52904494fa5ecf1ef594821fc P c15e23a34059456788f9775031e27ec0 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20240502 08:00:41.889114 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:41.897023 20606 consensus_peers.cc:485] T 1bc5937510214c3da7731d3ad2a67842 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20240502 08:00:41.911607 20606 consensus_peers.cc:485] T bbae35bae8c047a6982e128ffb77ca86 P b443e68b6212450286869124684bdfff -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20240502 08:00:41.921627 20078 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111) [suppressed 31 similar messages]
W20240502 08:00:41.928022 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:41.936686 20870 consensus_peers.cc:485] T 15bb9480e88b41219cc91168cc438c70 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Couldn't send request to peer ff9f2ccccdfa48a1afbc0e371f41c662. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
W20240502 08:00:41.954330 20474 consensus_peers.cc:485] T fb0f04b72dc046c0afee40b895e17a1d P e951f009258e473cb57cf8773c7975d4 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20240502 08:00:42.171833 20342 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111) [suppressed 40 similar messages]
W20240502 08:00:42.172396 20342 consensus_peers.cc:485] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20240502 08:00:42.293473 20870 consensus_peers.cc:462] T 1453e833d0ad494583c44b5496088f6d P b5f471f8b5944986a7d878cf6ac550b8 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111)
W20240502 08:00:42.318780 20210 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111) [suppressed 30 similar messages]
W20240502 08:00:42.362998 20870 consensus_peers.cc:462] T f588a3ec49a24b1a9a9e19cdf8c30ca5 P b5f471f8b5944986a7d878cf6ac550b8 -> Peer ff9f2ccccdfa48a1afbc0e371f41c662 (127.3.139.74:45559): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.3.139.74:45559: connect: Connection refused (error 111)
W20240502 08:00:42.408977 20342 consensus_peers.cc:485] T 04659b5e3504411987dafc49a5fa5ea7 P 35642aa2099c40e59bc2092604078e07 -> Peer f014c4ad4c1342219a1004181ac33ed8 (127.3.139.72:43861): Couldn't send request to peer f014c4ad4c1342219a1004181ac33ed8. Status: Network error: Client connection negotiation failed: client connection to 127.3.139.72:43861: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
W20240502 08:00:42.494359 20342 consensus_peers.cc:462] T e61fe3cb61c240f9b2c898aff1951144 P 35642aa2099c40e59bc2092604078e07 ->